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Abstract 


/  - 

With  the  enactment  of  Public  Laws  (P.L.  98-212  and  98- 
525)  warranties  must  be  considered  for  DOD  weapon  systems. 

It  is  DOD ' s  policy  to  only  obtain  cost  effective  warranties. 
To  determine  if  a  warranty  is  cost  effective  a  cost-benefit 
analysis  is  required.  There  has  been  concern  by  Congress, 
the  General  Accounting  Office,  and  the  Air  Force  about  the 
adequacy  of  the  cost-benefit  analyses.  This  theais  effort 
examined  the  nature  and  availability  of  information  needed 
to  conduct  cost-benefit  analysis  of  warranties. 

A  theoretical  model  of  warranty  relationships  was 
described  and  used  to  identify  the  information  required  to 
assess  the  benefits  and  costs  of  warranties.  In  order  to 
limit  the  scope  of  the  research,  only  jet  engine  warranties 
were  considered.  This  thesis'- effort  is  useful  in  that  a 
structure  for  warranty  cost-benefit  analysis  is  provided. 

In  addition,  several  problem  areas  involved  in  evaluating 
and  measuring  warranty  costs  and  benefits  are  identified  and 
discussed.  <.<*{•  >S.  /  . 


THE  FEASIBILITY  OF  A 
COST-EFFECTIVENESS  ASSESSMENT 
OF  AIR  FORCE  JET  ENGINE  WARRANTIES 

I .  Introduction 

General  Issue 

Procurement  decisions  in  the  Department  of  Defense 
(DOD)  are  made  by  managers  who  try  to  obtain  quality 
products  at  the  least  total  cost.  These  decisions  have  been 
hampered  by  the  decline  in  quality  (49:6,7)  and  the  ever- 
tightening  budget  constraints.  For  over  a  decade  Congress, 
users  of  equipment,  and  the  taxpaying  public  have  been 
concerned  with  the  following  four  problems.  One,  that  the 
defense  equipment  does  not  work  as  advertised.  Two,  that 
the  equipment  takes  too  long  to  develop  and  produce.  Three, 
that  the  equipment  cannot  be  repaired  once  it  is  in  the 
field,  and  if  it  can  be  repaired,  spare  parts  are  not 
available.  Four,  that  the  life  cycle  cost  of  developing, 
producing,  and  maintaining  the  equipment  is  too  high.  With 
the  intent  of  addressing  these  concerns  various  contracting 
initiatives  have  emerged. 

One  of  these  initiatives  has  been  the  use  of 
warranties.  Specifically,  warranties  are  used  as  a  method 
for  motivating  contractors  to  assure  that  the  products 
delivered  will  work  as  advertised;  and  also,  for  reducing 
life  cycle  cost  by  decreasing  operations,  maintenance,  and 
support  costs  (64). 
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until  fiscal  year  1984  and  1985  Congressional  action, 
warranties  were  used  in  military  production  contracts  on  a 
selective  basis.  However,  with  the  passage  of  Section  794 
of  the  1984  DOD  Appropriation  Act  (Public  Law  98-212)  and 
Section  2403  of  the  1985  DOD  Authorization  Act  (Public  Law 
98-525)  the  Department  of  Defense  is  required  to  obtain  cost 
effective  warranties.  Specifically,  three  warranties  (one 
covering  design  and  manufacturing  requirements,  one  covering 
defects  in  materials  and  workmanship,  and  one  covering 
essential  performance  requirements)  are  required  in  the 
procurement  of  weapon  systems  with  total  cost  exceeding  $10 
million  or  with  a  unit  cost  exceeding  $100,000  (9:6,7; 
23:11-2;  49:47).  If  the  warranty  is  not  cost  effective 
Congress  must  be  notified  (23:11-13,  II-22-II-25;  49:46; 
77:386) . 

To  determine  if  a  warranty  is  cost  effective,  DOD 
guidance  requires,  prior  to  procurement,  a  cost-benefit 
analysis  be  conducted  (23:11-26;  13:2-3;  49:57;  77:386).  In 
January  1985  a  change  to  the  Department  of  Defense  Federal 
Acquisition  Regulation  Supplement  provided  interim  guidance 
for  the  cost-benefit  analysis  (32).  This  guidance  was 
superseded  in  May  1986  with  final  guidance  that  expanded  the 
discussion  of  what  is  to  be  included  in  the  analysis  of 
proposed  warranties.  Specifically,  paragraph  246.770-8 
describes  the  cost-benefit  analysis  that  is  required: 

In  assessing  the  cost  effectiveness  of  a  proposed 

warranty,  an  analysis  must  be  performed  which  considers 
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both  the  quantitative  and  qualitative  costs  and 
benefits  of  the  warranty.  Costs  include  the  warranty 
acquisition,  administration,  enforcement  and  user 
costs,  weapon  system  life  cycle  costs  with  and  without 
a  warranty,  and  any  costs  resulting  from  limitations 
imposed  by  the  warranty  provisions.  Costs  incurred 
during  development  specifically  for  the  purpose  of 
reducing  production  warranty  risks  should  also  be 
considered.  Similarly,  the  cost-benefit  analysis  must 
also  consider  logistical/operational  benefits  expected 
as  a  result  of  the  warranty  as  well  as  the  impact  of 
the  additional  contractor  motivation  provided  by  the 
warranty  (41:947). 

Even  with  the  required  cost-benefit  analysis,  both 
Congress  and  the  General  Accounting  Office  have  been 
concerned  about  the  adequacy  of  the  warranty  analyses  (21; 
23:11-26).  In  addition,  cost-benefit  analysis  concerns  were 
documented  in  a  1985  Air  Force  Institute  of  Technology 
masters  thesis  by  Captains  Hernandez  and  Daney, 

USAF  people  did  not  know  what  type  of 

approach/structure  to  use  for  a  warranty  cost-benefit 
analysis.  Instead  the  USAF  people  we  interviewed 
relied  heavily  on  the  contractor  to  price  the  warranty 
and  then  analyzed  the  contractor’s  methodology  instead 
of  preparing  an  independent  estimate  for  comparison 
purposes.  This  is  a  potentially  serious  problem  since 
the  heart  of  determining  warranty  cost  effectiveness 
and  affordability  should  be  the  cost-benefit  analysis, 
which  from  our  research  was  not  being  accomplished 
independently  (49:63). 

Also,  interest  was  documented  in  a  January  1986  report 
prepared  by  the  Air  Force  Product  Performance  Agreement 
Center, 

Questions  have  been  raised  regarding  the  services' 
compliance  with  the  need  [requirement]  to 
determine  cost-effectiveness  of  warrant ies ...  a  need 
exists  to  examine  the  Air  Force  approach  to  warranty 
implementation  with  particular  emphasis  on  cost-benefit 
analysis  and  related  concerns  (9:6). 


The  purpose  of  this  research  is  to  examine  the  nature 
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and  availability  of  information  needed  to  conduct  cost- 
benefit  analyses  of  warranties.  The  remainder  of  this 
chapter  will  define  the  term  "warranty"  and  describe  the 
various  types  of  warranties,  review  the  Department  of 
Defense  warranty  history,  and  explain  the  plan  of  this 
thesis . 


Definition  of  Warranty 

For  the  purpose  of  this  research,  the  term  warranty 
will  be  defined  using  the  Federal  Acquisition  Regulation 
definition.  Warranties  can  also  be  segregated  by  types. 

Warranty  Defined .  A  warranty  is  a  legal  link  between 
the  buyer  and  the  seller  either  after  a  contract  is 
completed  or  while  the  contract  is  in  progress  (11:9). 
Warranty  is  defined  by  the  Federal  Acquisition  Regulation  as 
"a  promise  of  affirmation  given  by  a  contractor  to  the 
government  regarding  the  nature,  usefulness,  or  condition  of 
the  supplies  or  performance  of  services  furnished  under  the 
contract"  (33:46-9). 

Types  of  Warranties .  There  are  two  types  of 
warranties,  implied  warranties  and  expressed  warranties 
(11:10-11;  68:14-15).  John  Rannenberg  in  his  1984  Naval 
Postgraduate  School  masters  thesis.  Warranties  in  Defense 
Acquisition:  the  Concept,  the  Context ,  and  the  Congress , 

stated,  "implied  warranties  are  'read  in'  contracts  by 
common  law,  even  if  the  specific  language  is  not  addressed" 


ws * 


m 


M 


I'iH 


VA 


»v« 
■  Vf  1 

Vt*l 

■d 


P 


,l%] 


(68:14).  Expressed  warranties  are  where  "the  seller 


warrants  that  seller-designed  system,  accessories, 


equipment,  and  parts  shall  be  free  from  defects  in  design, 


material  and  workmanship  and  shall  conform  to  the  detailed 


specification  requirements  over  some  specified  period  of 


time"  (68:15) . 


Six  kinds  of  expressed  warranties  are  used  in  the 


Department  of  Defense: 


1)  Commercial  Warranties  -  the  contractor  determines 


the  terms  of  the  liability  and  the  responsibility  for 


correction. 


2)  Construction  Warranty  -  the  contractor  agrees  to 


correct  any  defect  in  design,  material,  or  workmanship  and 


any  nonconformance  of  work  to  meet  the  contract 


specifications  (11:12) 


3)  Correction  of  Deficiencies  Clause  -  the  contractor 


agrees  to  correct  any  design,  material,  or  workmanship 


deficiencies  which  result  when  the  specific  item  fails  to 


meet  specification  and  contractual  requirements  (11:11) 


4)  Reliability  Improvement  Warranty  (RIW)  -  the 


contractor  agrees,  for  a  specified  or  measured  period  of 


use,  to  repair  or  replace  defects  (subject  to  specified 


exclusions  if  applicable);  also,  the  contractor  may  be 


required  to  enhance  the  production  design  and  engineering  of 


the  system  or  equipment  so  as  to  improve  the 


field/operational  reliability  and  maintainability  (11:12). 


5)  Service  Warranty  -  the  contractor  agrees  to  correct 
services  if  defects  in  design  or  workmanship  are  found  prior 
to  the  expiration  of  a  specific  event  (11:12). 

6)  Supply  Warranty  -  the  contractor  is  responsible  to 
replace  or  rework  contract  items  if  defects  in  design, 
material  or  workmanship  are  found  prior  to  the  expiration  of 
a  specified  period  of  time  or  before  occurrence  of  a 
specific  event  (11:11). 

Department  of  Defense  Warranty  History 

Warranties  have  been  used  in  defense  acquisitions  for 
many  years.  During  the  history  of  the  use  of  warranties  in 
the  DOD  some  significant  events  and  turning  points  have 
occurred.  The  significant  events  can  be  divided  into  three 
timeframes:  prior  to  1964,  from  1964  through  the  1970's,  and 
in  the  1980's. 

Prior  to  1964.  Prior  to  the  mid-1960's  warranties  were 
frequently  used  in  government  contracts  (16:4;  45:25;  47:1). 
However,  each  military  department  of  the  DOD  had  its  own 
policies  (47:34)  and  its  own  version  of  warranty  clauses 
(16:4;  45:25;  47:22).  In  general,  warranties  were  used  to 
extend  the  contractor's  product  liability  beyond  initial 
acceptance  by  the  Government.  Specifically,  warranties  were 
used  to: 

1)  protect  the  Government  where  inspection  and 
acceptance  test  could  not  assure  there  was  full  conformance 
to  the  specification  (47:50), 
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2)  define  the  extent  of  the  contractor's  liability 
for  latent  defects  (47:50), 

3)  give  the  Government  time  to  conduct  the 
necessary  tests  required  for  some  complex  items  (47:51),  and 

4)  assure  that  contract  performance  would  be 
accomplished  (47:51). 

Fr°in  i^64  Through  the  1970 '  s .  During  the  timeframe 
from  1964  through  the  1970' s,  three  significant  events 
occurred  relating  to  warranties.  One  was  that  warranties 
were  included  in  DOD  procurement  regulations.  The  second, 
was  that  warranty  risk  issues  emerged,  resulting  in  DOD's 
policy  of  self  insuring.  Finally,  Reliability  Improvement 
Warranties  (RIW's)  were  introduced.  These  events  are 
discussed  below. 

Warranties  Appear  in  Regulation.  In  1964,  for  the 
first  time,  the  Armed  Services  Procurement  Regulation  (now 
the  Federal  Acquisition  Regulation)  established  uniform  DOD 
policy  for  the  use  of  warranties.  A  comprehensive  list  of 
instructions  for  the  use  of  warranties  in  fixed  price  type 
contracts  was  published  (45:25).  Warranties  were  to  be  used 
only  where  they  could  be  enforced  and  where  the  protection 
for  the  Government  was  greater  than  the  potential  increase 
in  contract  price.  Several  expansions  and  clarifications 
have  been  made  through  the  years  since  the  introduction  of 
warranties  in  the  DOD  regulation  (45:25;  13:2-2). 


POD  as  Self  Insurer .  In  the  late  1960‘s  expanded 
warranty  coverage  policies  added  new  issues  relative  to  the 
government  and  contractor  sharing  of  risk.  Specifically,  a 
"Warranty  of  Construction"  clause  made  the  contractor  liable 
for  "Consequential  Damages"  to  government  property.  An 
incident  arose  when  the  Austrialian  Government  sued  Lockheed 
Aircraft  and  its  subcontractor,  Menasco,  for  the  loss  of  an 
aircraft  and  other  damages  due  to  an  alledged  landing  gear 
failure.  An  out-of-court  settlement  resulted  in  Lockheed, 
Menasco,  and  the  United  States  Government  contributing  to 
the  price  of  a  replacement  aircraft.  The  Austrialian 
incident  led  to  a  review  of  DOD's  procurement  policy  on 
consequential  damages  and  warranties  (23:11-1,11-2;  31:D-1; 
45:25)  . 

Subsequently  in  1971,  DOD  announced  a  procurement 
policy  change  aimed  at  reducing  costs  by  limiting  the 
contractor's  risk  (31:D-1;  45:25).  The  change  provided, 
that  DOD  would  generally  "act  as  a  self-insurer  for  loss  of 
or  damage  to  property  of  the  government  occurring  after 
final  acceptance  of  supplies  delivered  to  the  government  and 
resulting  from  any  defects  or  deficiencies  in  such  supplies" 
(45:25-26).  Based  on  DOD's  liability  policy  as  a  self 
insurer,  the  use  of  warranties  were  selectively  applied  to 
government  acquisitions. 

Introduction  of  RIW.  In  the  mid-1960's,  an 
innovative  warranty,  "Failure  Free  Warranty",  was  originally 


proposed  by  the  Instrument  Division  of  Lear  Siegler,  Inc.  to 
the  Air  Force  (11:31;  12:1).  The  "Failure  Free  Warranty" 
was  different  from  previous  warranties  in  that  (1)  all  field 
failures  were  covered  under  warranty  for  an  extended  time 
period,  (2)  risk  was  shifted  so  the  contractor  shared  any 
reward  (expanded  profit)  or  penalty  (fixed  price)  with  the 
government,  and  (3)  reliability  was  measured  in  field 
operation  after  hardware  acceptance  (50:17).  Although  the 
proposal  was  not  accepted,  the  ground  work  was  laid  for  what 
would  later  be  referred  to  as  a  Reliability  Improvement 
Warranty  (50:17). 

In  the  late  1960's  and  into  the  early  1970's,  several 
"Failure  Free  Warranty"  contracts  were  awarded  by  both  the 
Navy  and  the  Air  Force  (11:31;  12:1;  16:4,5;  13:2-2;  53:26). 
The  contracts  were  for  equipment  that  already  existed  in  the 
field,  which  minimized  risk  for  both  the  contractor  and  the 
Government . 

In  1973,  1974,  and  1975  DOD  memorandums  were  sent  to 
the  service  secretaries  establishing  the  use  of  Reliability 
Improvement  Warranties  for  electronic  systems  and  providing 
guidelines  for  their  application.  The  1974  memo  reiterated 
that  the  intent  of  the  RIW  contracting  technique  was  to 
realize  improved  operational  reliability  and  maintainability 
of  DOD  systems  and  equipments  for  each  additional  dollar 
that  the  contractor  uses  (50:7;  19:1,2). 


The  1975  memo  resulted  from  industries  comments  to 
clarify  and  expand  DOD's  RIW  guidelines.  Discussions  were 
held  between  DOD  and  industry  in  which  internal  and  external 
experiences  were  used  to  "refine"  DOD's  continued 
application  of  warranties  (45:27). 

The  result  of  warranties  being  included  in  procurement 
regulations,  DOD's  policy  of  self  insuring,  and  the 
introduction  of  RIW's  was  that  warranties  were  selectively 
used.  By  1979,  warranties  were  estimated  to  cover  one-third 
of  the  4.1  million  types  of  items  in  DOD  inventory  (45:26). 
In  addition,  about  $15  billion  of  DOD's  annual  procurements 
for  material  included  some  type  of  warranty  coverage  at  an 
annual  cost  of  $300  million  (70:1). 

In  the  1980 1 s .  During  the  1980 's  significant 
occurrences  related  to  warranties  continued.  Three  of  these 
events  are  discussed.  One  is  that  the  Air  Force  took  an 
innovative  interest  in  warranties.  The  second  is  that 
warranties  were  included  as  part  of  the  Defense  Acquisition 
Improvement  Program  (DAIP) .  Third,  Congressional  laws  were 
enacted  pertaining  to  warranties. 

Air  Force  Innovations .  In  August  and  December  of 
1979,  the  Air  Force  and  industry  reviewed  commercial  type 
warranties  for  application  to  military  hardware  (45:27). 
There  was  a  movement  to  change  the  term  "warranty"  to  the 
term  "product  performance  agreement"  which  broadened  the 
review  and  application.  The  result  of  these  meetings,  was  a 


jointly  developed  set  of  principles  to  guide  the  efforts  of 
fielding  a  quality  product  (45:27).  The  principles  were 
subsequently  published  by  the  Air  Force  in  a  Product 
Performance  Agreement  Guide  (45:27).  The  Guide  described  a 
number  of  warranty  provisions  with  their  applications, 
advantages,  and  disadvantages  (68:61). 

In  December  1980,  the  Air  Force  initiated  an  effort  to 
establish  the  “Product  Performance  Agreement  Center"  at 
Wright-Patterson  Air  Force  Base,  Ohio.  The  Center  was  to 
serve  as  a  DOD-Industry  clearing  house  for  product 
performance  data  and  analysis  (68:61).  The  contractor's 
effort  in  the  solicitation  included  updating  the  Product 
Performance  Agreement  Guide,  risk/cost  benefit  modeling,  and 
general  administrative  support  (68:61).  On  27  September 
1982,  the  Air  Force  concluded  the  competitive  source 
selection  by  selecting  a  support  contractor  for  the 
government  operated  Center  (14:1-1). 

Warranties  as  Part  of  DAIP .  In  1981,  senior  level 
DOD  interest  in  warranties  was  seen  as  one  way  to  enhance 
the  acquisition  process.  Mr.  Carlucci,  then  the  Deputy 
Secretary  of  Defense,  recognized  the  importance  of 
reliability  in  reducing  life  cycle  cost  when  he  proposed  his 
32  initiatives  which  became  known  as  the  Defense  acquisition 
Improvement  Program  (18:5).  These  initiatives  were  later 
consolidated  in  1983  by  Mr.  Thayer  (Carlucci's  successor) 
into  3ix  initiatives  (18:5).  One  of  Thayer's  initiative, 


Improved  Support  and  Readiness,  contained  Carlucci's 
original  action  16  which  stated,  "...program  management 
office  [should]  employ  specific  contractual  incentives 
focused  on  designing  for  reliability  and  supportabi lity " 
(18:5,11;  31:D-2).  Included  in  these  contractual  incentives 
was  the  use  of  warranties  (67:1-3,1-4). 

Laws  Governing  Warranties .  In  1983,  Senator  Mark 
Andrews  sponsored  a  warranty  provision  as  an  amendment  to 
H.R.  4185,  the  fiscal  year  1984  DOD  Appropriation  Act 
(68:63).  At  a  meeting  with  industry  representatives,  the 
Senator  discussed  his  reasons  for  developing  the 
legislation.  In  part,  he  had  a  desire  to  mandate  a 
commercial  marketing  environment  upon  the  defense 
acquisition  process  where  both  parties  would  make  and 
enforce  stricter  commitments  (68:64).  The  amendment  became 
section  794  of  the  1984  DOD  Appropriation  Act,  Public  Law 
98-212.  It  provided  that  "no  funds  would  be  obligated  or 
expended  for  the  procurement  of  a  weapon  system  unless  the 
contractor  provided  certain  written  guarantees"  (1:37).  The 
key  features  were  (1)  emphasis  of  warranting  the  entire 
weapon  system;  (2)  having  the  prime  contractor  correct 
defects;  and  (3)  requiring  specific  performance  warranties 
(23:11-13-11-15;  49:42). 

The  new  law  left  some  questions  and  caused  concern  in 
both  industry  (23:11-19;  49:44)  and  DOD  (23:11-17-11-19; 
68:73-76).  It  was  felt,  by  both  proponents  and  opponents  of 


the  warranty  legislation  that  a  rewritten  version  was 
necessary  (49:44;  68:77).  On  31  May  1984,  a  compromise 
effort  was  published  as  Section  191  of  the  Defense 
Authorization  Bill,  S.  2414  (68:77).  The  bill  was  passed  on 
19  October  1984  as  Public  Law  98-525,  Section  2403  of  the 
1985  Defense  Authorization  Act  (49:44). 

Public  Law  98-525  required  three  specific  kinds  of 
warranties  to  be  written  in  contracts  awarded  for  the 
production  of  weapon  systems.  The  law  applied  to  programs 
with  the  total  cost  exceeding  $10  million  or  with  a  unit 
cost  exceeding  $100,000  (9:6,7;  23:11-22;  49:47,48).  The 
three  kinds  of  warranties  were  described  in  a  report 
prepared  by  the  Air  Force  Product  Performance  Agreement 
Center: 

1)  Material/Workmanship  warranty  requires  that  the  item 
contracted  for,  at  time  of  delivery,  will  be  free  from  all 
defects  in  material  and  workmanship  (9:6). 

2)  Design/Manufacturing  warranty  requires  that  the  item 
will  conform  to  the  design  and  manufacturing  specifications 
of  the  contract  (9:6). 

3)  Performance  warranty  requires  that  the  item  will 
conform  to  specific  performance  criteria  under  specific 
conditions.  This  warranty  only  applies  after  manufacture  of 
either  the  first  ten  percent  of  the  eventual  total 
production  or  the  initial  production  quantity,  whichever  is 


Plan  of  This  Thesis 

The  purpose  of  this  research  is  to  examine  the  nature 
and  availability  of  information  needed  to  conduct  cost- 
benefit  analyses  of  warranties.  In  order  to  limit  the  scope 
of  the  research,  only  jet  engine  warranties  will  be 
considered.  Chapter  2  makes  reference  to  a  theoretical 
model  of  warranty  relationships  in  order  to  identify  the 
information  required  for  cost-benefit  analysis  of 
warranties.  Chapter  3  presents  an  overview  of  the  history 
of  Air  Force  engine  warranties.  Also  in  Chapter  3  the  jet 
engine  warranty  environment,  particularly,  selected  Air 
Force  procedures  relating  to  engine  warranties  are 
presented.  In  addition,  the  specific  engines  used  in  the 
research  are  identified.  In  chapter  4,  the  feasibility  of 
obtaining  the  information  required  for  assessing  the  costs 
and  benefits  of  engine  warranties  will  be  discussed.  The 
nature  and  availability  of  the  information  required  along 
with  the  problems  involved  with  measuring  the  costs  and 
benefits  of  engine  warranties  are  considered.  Chapter  5 
presents  the  summary,  conclusions,  and  recommendations  of 
this  thesis  effort. 
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The  purpose  of  this  chapter  is  to  identify  the 
information  required  in  order  to  assess  the  benefits  and 
costs  of  warranties.  To  do  this,  extensive  reference  will 
be  made  to  the  working  paper,  "Evaluating  the  Benefits  and 
Costs  of  DOD  Warranties,"  by  Professor  Leroy  Gill  of  the  Air 
Force  Institute  of  Technology.  This  chapter  begins  first  by 
considering  the  nature  of  costs  and  benefits.  Next  the 
purposes  of  commercial  and  DOD  warranties  are  contrasted, 
and  the  insurance  and  assurance  aspects  of  warranties  are 
separated.  Then  the  impact  of  warranties  on  manufacturer 
incentives  is  considered.  Finally,  the  methodology  of 
evaluating  the  benefits  and  costs  of  DOD  warranties  is 
addressed . 

The  Nature  of  Costs  and  Benefits 

Nature  of  Costs .  Costs  are  the  value  of  the  relevant 
inputs  associated  with  producing  a  given  product  or  outcome. 
Relevant  means  that  the  costs  are  contingent  upon  the 
specific  decision  or  choice  being  made.  In  this  sense,  the 
term  cost  differs  from  its  standard  usage.  The  relevant 
costs,  used  to  decide  between  alternatives,  are  costs  that 
lie  in  the  future  not  in  the  past  (42:33).  Relevant  costs 
do  not  include  "sunk"  cost  and  do  include  "opportunity" 


costs . 


"Sunk"  costs  are  past  costs.  They  are  costs  that  have 
been  incurred  as  the  result  of  a  previous  decision  (36:20; 
42:33).  Sunk  costs  can  not  be  retrieved,  therefore  they  are 
not  relevant  because  they  are  beyond  the  control  of  the 
current  decision  or  choice  being  made.  Relevant  costs  are 
avoidable  costs.  "Opportunity"  costs  are  the  value  of  the 
alternative  given  up  as  a  result  of  the  current  decision 
(42:25;  44:49).  For  ne»r  items,  opportunity  costs  are  equal 
to  their  purchase  price.  For  previously  purchased  items, 
their  opportunity  cost  is  equal  to  their  current  resale 
value. 

Nature  of  Benefits .  Benefits  are  the  value  of  the 
output  produced.  In  order  to  assess  benefits  it  is 
necessary  to  identify  and  classify  outputs.  The  categories 
for  the  outputs  should  be  defined/described  in  accordance 
with  the  requirements  of  the  decision.  The  characteristics 
of  the  categories  include  all  relevant  benefits  in  relation 
to  the  objective  of  the  alternative  (36:23).  The  benefits 
may  be  quantitative  or  qualitative.  Ideally,  they  should  be 
discretly  identifiable  and  measurable. 

Commercial  Warranties  and  POD  Warranties 

Commercial  Warranties .  In  the  private  sector, 
competitive  conditions  encourage  manufacturers  to  offer 
warranties  (60:2).  The  warranties  make  a  statement  that  the 
manufacturer  will  stand  behind  its  product,  and  assume  the 
risk  in  case  the  product  fails  to  perform  as  promised.  Many 
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manufacturers  offer  warranties  so  that  customers  gain 
confidence  that  the  product  quality  is  equal  to  the 
advertised  promises  (62:3). 


POD  Warranties .  In  DOD,  warranties  are  desirable  as  an 
incentive  to  control  Life  Cycle  Cost  and  as  a  method  to 
obtain  quality  products  (53:23;  64).  The  DOD  recognizes 
that  the  cost  of  a  system  includes  expenses  of  not  only 
acquiring  the  system,  but  also,  the  expenses  of  operating 
and  supporting  the  system.  Part  of  the  support  cost 
includes  equipment  failure  and  the  expense  involved  in 
repair/replacement  (53:24).  In  order  to  motivate  the 
manufacturer  to  consider  the  support  cost,  DOD  warrants 
performance  requirements  (53:26).  This  is  an  attempt  to 
improve  product  reliability,  by  providing  economic  sanctions 
(i.e.,  repair  or  replacement  of  the  product  by  the 
manufacturer)  if  the  product  fails  to  perform  as  intended 
(16:3;  60:2).  In  addition,  the  improved  reliability 
increases  readiness  by  having  more  equipment  available  for 
use  (43:8;  60:26;  73:2,3). 


Warranties  as  Insurance  and  as  Assurance 

Both  the  commercial  and  DOD  customer  can  view 
warranties  from  two  perspectives  (73:5).  First,  the 
warranty  as  insurance  in  that  the  customer  does  not  have  to 
pay  for  the  repair/replacement  of  a  failure  during  the 
period  of  the  warranty  (43:1;  60:3).  Second,  the  warranty 
as  assurance  in  that  the  customer  is  receiving  a  quality 


product  (43:1;  60:3) 


warranty  as  Insurance .  The  insurance  perspective  can 
be  referred  to  as  the  direct  advantage  of  a  warranty  for  the 
purchaser  (60:3).  If  the  product  purchased  does  not  perforin 
as  intended,  the  purchaser  receives  compensation  from  the 
seller.  The  attractiveness  of  warranties  from  this 
perspective  can  be  explained  by  looking  at  the  concepts  of 
risk  and  insurance.  With  a  warranty,  the  risk  of  "loss"  is 
shifted  from  the  buyer  to  the  seller  for  a  specified  period 
of  time. 

Consider  an  example  given  by  Dr.  Gill  in  "Evaluating 
the  Benefits  and  Costs  of  DOD  Warranties,"  (43:2).  If  the 
annual  probability  to  an  individual  of  losing  his/her  home 
to  a  fire  is  1  in  100,000  and  the  value  of  the  home  is 
§100,000,  then  the  expected  loss  per  year  is  $1  (43:2). 
However,  in  reality,  the  homeowner  could  lose  as  much  as 
§100,000,  so,  for  protection,  the  homeowner  buys  an  annual 
fire  insurance  policy.  The  buyer  pays  a  premium  to  cover 
the  loss  of  his/her  home  in  case  a  fire  should  destroy  it 
during  the  year.  The  homeowner  is  shifting  the  risk  of  loss 
(replacing  the  home)  to  the  insurance  company. 

On  the  other  hand,  the  insurance  company  spreads  the 
potential  losses  over  a  large  number  of  insured  homes,  by 
pooling  the  risk  or  probability  of  a  loss  occurring.  For 
the  insurance  company,  by  insuring  a  large  number  of  homes, 
the  probability  that  all  the  homes  will  burn  down  is 
minuscule.  Therefore,  by  pooling  risk,  the  insurance 
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company  has  greatly  diminished  the  probability  of  a 
catastrophic  outcome  (43:2). 


For  the  DOD,  the  insurance  advantages  of  a  warranty  are 
more  likely  achieved  by  “self-insuring",  because  of  the 
large  number  of  like  items  in  DOD  inventory  for  which  risk 
of  loss  can  be  pooled  (43:2). 

Warranty  as  Assurance .  A  warranty  implies  the  seller 
is  producing  a  quality  product,  because  non-quality  products 
would  have  to  be  repaired/replaced  under  warranty  claims. 
Implicitly  it  is  reasoned,  that  warranty  claims  would  cost 
the  seller  money  in  repairing  and/or  replacing  the  non¬ 
quality  product  and  would  affect  the  manufacturer's  profit 
(43:2;  60:3).  It  is  also  reasoned,  that  with  a  warranty, 
the  cost  of  warranty  claims  is  more  than  the  incremental 
cost  associated  with  producing  a  product  of  the  advertised 
quality  (43:2,3). 

Impact  of  Warranty  on  Manufacturer  Incentives 

When  a  purchase  is  made  without  a  warranty,  there  is  no 
direct  incentive  for  the  manufacturer  to  make  his  product 
any  more  reliable  than  the  market  demands.  This  is  because, 
greater  reliability  increases  manufacturing  costs  and  such 
increased  costs  will  be  avoided  unless  market  conditions 
make  it  profitable  for  a  manufacturer  to  build  a  more 
reliable  product.  Under  a  warranty,  decreases  in 
reliability  of  the  product  means  increases  in  the  number  of 
warranty  claims  which  affect  the  manufacturer's  profit 
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(60:3).  Alternatively,  increases  in  reliability  of  the 
product  means  increases  in  manufacturing  costs  which  also 
affects  the  manufacturer's  profit  (53:24).  A  model  which 
shows  the  effect  of  the  warranty  on  the  manufacturer's  total 
cost  can  be  visualized  in  Figure  1  (43:5). 

Figure  1  describes  (for  a  fixed  number  of  units 
produced)  the  standard  case  of  the  manufacturer's  total  cost 
when  warranty  claim  cost  are  included  (43:7).  The  convex 
curve  assumption  is  used  to  illustrate  the  practical 
considerations  that  the  observed  percentage  of  reliable 
products  will  lie  between  zero  and  one  (43:7);  and  that,  as 
reliability  is  improved,  the  manufacturing  costs  increases 
at  an  increasing  rate  (43:7;  53:24).  Another  assumption  is 
that  the  warranty  requires  the  manufacturer  to  pay  a  fixed 
penalty  cost  per  failure  (43:3).  The  manufacturer's  total 
cost  is  obtained  by  vertically  adding  the  warranty  claim 
cost  and  the  manufacturing  cost  (43:4). 

The  warranty  claim  cost  is  equal  to  the  fixed  penalty 
cost  per  failure  times  the  number  of  failures  (43:4).  For 
example  in  Figure  1,  if  all  the  products  produced  failed, 
the  manufacturer's  potential  warranty  claim  cost  would  be 
equal  to  E.  On  the  other  hand,  if  there  are  no  failures, 
the  warranty  claim  cost  would  be  equal  to  zero. 

The  manufacturing  cost  is  a  function  of  (1)  the 
quantity  of  the  products  produced  (for  Figure  1  the  quantity 
is  fixed),  and  (2)  the  reliability  of  those  products 


produced  (43:4).  If  the  given  quantity  of  products  in 
Figure  1  have  a  zero  reliability,  the  manufacturing  cost 
would  be  equal  to  C.  However,  if  the  products  are  100 
percent  reliable,  the  manufacturing  cost  would  be  B. 

The  standard  assumption  is  that  the  manufacturer  is  a 
profit  maximizer.  A  corollary,  is  that  the  manufacturer 
will  tend  to  reduce  total  cost,  or  be  a  cost  minimizer  for 
any  given  level  of  output.  Therefore,  in  Figure  1  with  the 
level  of  output  (and  revenue)  fixed,  the  manufacturer  will 
strive  for  the  cost  which  corresponds  to  the  minimum  point 
on  the  total  cost  curve  and  is  associated  with  the 
reliability  of  Z'  (43:6). 

The  effect  of  an  increase  in  warranty  coverage  is 
graphically  shown  in  Figure  2.  An  increase  in  warranty 
coverage  makes  the  cost  of  failures  more  expensive,  thus 
increasing  the  fixed  warranty  cost  per  failure.  The  shift 
in  the  warranty  claim  cost  slope,  changes  the  manufacturer's 
total  cost  curve  (43:9).  The  change  is  shown  in  Figure  2 
where  the  solid  lines  are  the  new  cost  curves  and  the  dashed 
lines  are  the  cost  curves  from  Figure  1.  The  minimum  point 
on  the  total  cost  curve  also  shifts  and  the  associated 
reliability  increases.  In  Figure  2  the  increased 
reliability  is  annotated  as  Z1'.  Therefore,  increases  in 
warranty  cost  per  failure  (or  warranty  claim  penalties) 
motivates  the  manufacturer  to  produce  units  of  higher 
reliability  (43:7). 


In  reality,  profit  maximizing  solutions  may  be  bounded 
so  that  the  reliability  can  not  fall  below  a  given  positive 
level  due  to  political  or  market  considerations;  and  also, 
reliability  can  not  rise  above  a  maximum  level  due  to 
technological  constraints  (43:7).  The  effect  of  an  increase 
in  warranty  coverage  can  also  be  shown  mathematically.  The 
interested  reader  is  referred  to  Dr.  Gill's  paper, 
"Evaluating  the  Benefits  and  Costs  of  DOD  Warranties" 

(43:15)  . 

Using  the  simple  models  in  Figures  1  and  2,  it  can  be 
seen  that  the  level  of  warranty  coverage  influences  the 
manufacturer's  total  cost  and  can  motivate  the  manufacturer 
to  produce  more  reliable  products.  Warranties  affect  the 
manufacturer's  total  cost  by  influencing  the  warranty  claim 
costs  and  the  manufacturing  costs  (43:12).  The  manufacturer 
makes  a  trade-off  between  these  two  costs.  For  example,  as 
warranty  claim  costs  increases  the  manufacturing  costs 
decreases  and  vice-a-versa  (43:12).  Analysis  of  the  models 
shows  that  increasing  the  cost  of  failure  to  the 
manufacturer,  the  warranty  claim  cost,  increases  the  total 
manufacturing  cost  and  provides  an  incentive  for  the 
manufacturer  to  build  a  more  reliable  product  (43:12). 

Defining  the  Benefits  and  Costs  of  DOD  Warranties 

DOD  warranties  represent  a  trade-off  between  the 
benefits  and  the  costs  of  improving  reliability.  These 
trade-offs  can  be  evaluated  using  cost-benefit  analysis.  In 


order  to  make  effective  comparisons  of  alternatives,  it  is 
necessary  to  carefully  separate  costs  from  benefits  (43:7). 
The  following  considerations  should  be  made  when  defining 
the  benefits  and  costs  of  DOD  warranties. 

Defining  the  Benefits  of  DOD  Warranties .  Before 
describing  what  the  benefits  are,  let's  look  at  what  they 
are  not.  Benefits  are  not  cost  savings  (43:7).  For 
example,  if,  as  a  result  of  a  warranty,  there  are  reductions 
in  certain  costs,  such  as  fewer  repair  parts  purchased, 
these  advantages  are  measured  when  comparing  total  costs  of 
the  different  alternatives  (43:8).  The  repair  parts  for  the 
warranted  system  are  reduced  or  eliminated  in  comparison  to 
the  cost  of  operating  a  non-warranted  system.  Therefore, 
the  "cost-savings"  of  purchasing  less  repair  parts  should 
not  be  considered  a  "benefit"  of  the  warranty. 

DOD  benefits  relate  to  mission  performance  and  are 
improvements  in  readiness  as  a  result  of  having  more 
reliable  equipment  available  for  use  (43:8;  60:26;  73:2-3). 
Reliability  estimates  are  important  to  DOD  analyses  of 
benefits  (43:8).  When  comparing  alternatives, 
considerations  should  be  given  to  changes  in  reliability 
that  are  accomplished  by  warranties  and  other  methods  (i.e., 
increased  preventive  maintenance;  extended  development; 
intensive  testing;  design  redundancy;  etc.)  (43:8;  73:4). 

The  benefits  of  a  warranty  are  the  value  of  the 
increased  performance  as  result  of  a  warranty  when  compared 


to  the  other  alternatives.  When  evaluating  warranty 
benefits,  the  important  question  concerns  the  chanae  in 
reliability  as  a  result  of  the  change  in  warranty  coverage 
(43:8) . 

Defining  the  Costs  of  POD  Warranties .  The  costs  of 
warranties  can  be  classified  as  explicit  cost  and  implicit 
cost.  The  explicit  cost  of  DOD  warranties  are  the 
manufacturer's  total  cost  for  a  given  reliability  level  plus 
the  manufacturer's  profit.  Because  the  manufacturer's 
profit  is  included  in  the  cost  of  the  DOD  warranty,  it  might 
be  possible  for  DOD  to  obtain  improved  warranty  coverage  at 
no  additional  cost,  but  instead,  at  a  reduction  of  the 
manufacturer's  profits  (43:9;  77:387).  What  the  DOD  pays 
for  the  warranty  will  depend  on  the  negotiating  power  of  the 
DOD  with  the  manufacturer  (43:9).  The  DOD  should  not  pay 
more  for  a  change  in  reliability  than  the  value  of  the 
benefit.  Also,  it  should  be  recognized,  the  manufacturer's 
minimally  acceptable  increase  in  price,  for  a  change  in 
warranty  coverage,  is  likely  to  be  the  change  in  the 
manufacturer's  total  cost;  since  any  lesser  price  would 
reduce  the  manufacturer's  profit  from  previous  levels 
(43:10) . 

In  order  to  negotiate  intelligently  for  a  warranty  or  a 
change  in  warranty  coverage,  it  is  important  for  DOD  to 
estimate  the  increased  product  reliability,  and  also,  the 
impact  on  the  manufacturer's  total  cost  (43:10). 


For  DOD  to  estimate  the  cost  of  a  warranty  or  a  change 
in  warranty  coverage  for  the  manufacturer,  several  values 
need  to  be  known.  DOD  needs  to  know  the  manufacturing  cost 
of  the  product,  the  initial  level  of  reliability,  the  change 
in  reliability,  and  the  penalty  cost  per  warranty  claim 
(43:10).  Estimates  of  the  manufacturing  cost  require 
detailed  engineering  and  financial  information  (43:10). 
Estimates  of  the  reliability  and  the  warranty  claim  cost 
requires  knowledge  of  the  number  of  failures  before  and 
after  the  warranty,  and  also,  an  estimate  of  the 
contractor's  cost  to  repair/replace  the  product  in 
accordance  with  the  warranty  contract  provision(s)  (43:10). 

Implicit  costs  to  DOD  are  the  costs  necessary  to 
enforce  the  warranty  coverage.  These  implicit  costs  are 
costs  internal  to  DOD  that  are  necessary  to  enforce  legal 
warranty  claims  (i.e.,  administrative  costs)  (43:10). 

Unless  these  costs  and  the  actions  they  represent  are 
identified  and  controlled,  DOD  may  not  be  receiving  full 
value  of  the  warranty.  For  example,  if  one-half  of  the 
warranty  claims  are  not  filed  or  paid  because  of  inadequate 
administrative  costs  or  poor  procedures,  then  the 
manufacturer's  warranty  claim  cost  is  one-half  what  it 
should  be  (43:11).  The  result  could  be  either  the  DOD 
doesn't  have  reliable  products  available  for  missions,  or 
the  manufacturer,  in  order  to  increase  profits,  could  be 
producing  products  that  are  of  lesser  reliability  than  the 
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warranted  reliability  level. 

Implications  for  the  Evaluation  of  the  Benefits 
and  Costs  of  POD  Warranties 

From  a  theoretical  viewpoint,  there  are  several 
implications  with  regard  to  the  evaluation  of  the  benefits 
and  costs  of  DOD  warranties.  First,  for  benefits, 
warranties  increase  reliability.  In  order  to  evaluate  the 
benefits,  one  has  to  know  or  estimate  the  change  in 
reliability  which  results  from  a  warranty  and  the  value  of 
the  changed  reliability  in  an  operational  context  to  DOD 
(43:12)  . 

Second,  for  costs,  warranties  increase  the 
manufacture's  total  cost.  In  order  for  DOD  to  evaluate  the 
costs  one  has  to  know  as  much  as  possible  about  (a)  the 
actual  cost  (to  the  manufacturer)  of  a  failure;  (b)  the 
number  of  failures  which  occur  at  a  given  level  of  warranty 
coverage;  and  (c)  the  manufacturer's  cost  of  building 
reliability  into  the  product  (43:12,13). 

With  regard  to  the  actual  cost  (to  the  manufacturer)  of 
a  failure,  it  is  important  to  realize  that  the  actual  cost 
could  differ  from  implied  contractual  cost  because  of  the 
lack  of  adequate  documentation  or  violation  of  warranty 
provisions.  In  addition,  the  manufacturer  could  decrease 
his  actual  cost  for  a  failure  by  making  slow  or  incorrect 
repairs  and  by  requiring  excessive  documentation  (43:13). 


Finally,  to  evaluate  the  cost-effectiveness  of 
warranties,  the  benefits  and  costs  of  DOD  warranties  must  be 
compared  to  those  of  other  alternatives  (i.e.,  no  warranty, 
increased  maintenance,  etc.).  For  DOD,  the  benefits  of 
warranties  are  the  value  of  increases  in  reliability,  which 
increases  the  products  availability  for  use,  net  any 
limitations  on  use  of  the  product  and  any  reduction  in  in 
availability  (due  to  sending  the  product  back  to  the 
contractor  for  repair/replacement)  that  is  sometimes 
associated  with  warranties.  The  cost  of  warranties  include 
not  only  the  actual  contract  price,  but  also  the  internal 
cost  of  documenting  and  complying  with  the  claim  provisions 
(43:13).  These  costs  are  partially  offset  by  lower 
operating  and  support  costs  due  to  fewer  repairs  and  the 
costs  absorbed  by  the  contractor  who  repairs/replaces  the 
failed  units  during  the  warranty  period. 

In  order  for  warranties  to  assure  that  manufacturers 
deliver  products  that  will  work  as  advertised  at  the  least 
cost  for  the  government,  reliability  and  cost  information 
must  be  collected  and  used  in  evaluating  the  benefits  and 
costs  of  DOD  warranties. 

In  the  next  chapter  the  focus  will  be  on  jet  engine 
warranties,  particularly,  the  history  of  AF  engine 
warranties,  the  acquisition  of  engines  and  warranties,  and 
the  selection  of  engines  to  be  studied. 
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This  research  is  limited  to  a  consideration  of  the 
information  available  for  evaluating  the  costs  and  benefits 
jet  engine  warranties.  In  order  to  facilitate  the 
discussion,  this  chapter  provides  a  brief  history  of  engine 
warranties  in  the  Air  Force.  In  addition,  information  on 
Air  Force  procedures  pertaining  to  engines  is  given. 
Finally,  the  specific  engines  studied  are  identified. 


Past  History  of  Air  Force  Engine  Warranties 


Commercial  airlines  were  the  leaders  in  the  area  of 
aircraft  engine  warranties.  Commercial  airlines  received 
extensive  warranties  from  engine  manufacturers  for  a  number 
of  years  prior  to  the  same  type  of  warranties  being  applied 
to  Department  of  Defense  aircraft  engines  (34:8;  48:12; 
82:9).  The  engine  manufacturers  were  reluctant  to  provide 
other  than  limited  warranties  to  the  Air  Force;  because  they 
considered  it  too  risky  with  respect  to  the  military 
aircraft  operational  and  maintenance  environment  (34:10; 
48:13,14;  82:11). 

The  only  warranty  provision  that  was  widely  used  by  the 
Air  Force  for  engines  was  the  Correction  of  Deficiencies 
(COD)  clause,  which  required  the  contractor  to  correct  any 
condition  that  was  not  in  compliance  with  the  contract 
(34:10;  82:13).  The  limitation  for  the  Air  Force  was  that 
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the  COD  clause  only  covered  direct  or  primary  damage  (damage 
suffered  by  a  part  due  to  it  own  failure)  (82:13).  In 
addition,  it  was  difficult  to  prove  that  the  contractor  was 
at  fault  for  the  deficiency  (34:10). 

In  1972,  a  study  was  conducted  for  the  Assistant 
Secretary  of  Defense  for  Installations  and  Logistics  on 
"Methods  of  Acquiring  and  Maintaining  Aircraft  Engines." 
Among  the  conclusions  resulting  from  the  study  were  the 
following : 

The  aircraft  engine  warranty  has  been  successful  in 
circumstances  where  the  management,  maintenance,  and 
operation  of  the  engine  were  highly  controlled.  The 
usual  military  environment  does  not  permit  such  control 
(23:11-3). 

A  warranty  almost  always  reduces  life  cycle  support 
costs  by  reason  of  the  warranty  (34:11). 

The  study  recommended  "against  the  use  by  DOD  of  aircraft 

engine  warranties  at  this  time,"  and  that  "continued 

research  and  analysis  into  specific  application  of 

warranties  to  military  aircraft  engines  should  be 

encouraged"  (23:11-3). 

The  Air  Force  continued  to  assess  the  advantages  of 
engine  warranties  in  the  commercial  sector  and  their 
applicability  to  military  aircraft  engines  (23:11-3; 
34:11,12) . 

In  1978,  General  Alton  D.  Slay,  then  the  Commander  of 
Air  Force  Systems  Command,  directed  procurement  agencies  to 
explore  and  implement  useful  commercial  contracting 
practices  which  included  warranties  (48:2,15;  82:3).  From 


this  direction,  the  Deputy  for  Propulsion,  Aeronautical 
Systems  Division  initiated  a  number  of  studies  for  engine 
warranty  application  (48;2,16).  Out  of  the  initial 
research,  warranties  for  military  engines  were  requested  in 
Air  Force  engine  proposals  (23:11-3;  82:13). 

By  1980  the  Air  Force  had  a  few  warranties  on  jet 
engines.  Each  of  these  warranties  was  unique  and  designed 
to  meet  the  specific  needs  of  the  program  and  the  individual 
engine  (65).  As  time  went  on,  the  number  of  engine 
warranties  being  applied  increased,  in  part  due  to 
Congressional  action. 

In  1982,  Congressional  action  specifically  addressed 
the  warranty  of  the  Air  Force's  alternate  fighter  engine 
with  the  1983  Defense  Appropriation  Act,  Section  797  of 
Public  Law  97-377.  Briefly,  the  act  stated  that  no  funds 
would  be  provided  to  purchase  the  alternate  fighter  engines 
unless  a  specific  warranty  was  written  in  the  contract  (21; 
23:11-4,  I 1-6 ) . 

The  evolution  of  engine  warranties  resulted  in  two 
documents,  the  Model  Engine  Warranty  and  the  Joint  Engine 
Warranty  Development  Guide .  In  1981,  the  Model  Engine 
Warranty  was  developed  to  reduce  the  effort  required  to 
create  an  engine  warranty  (82:19).  The  model,  influenced  by 
commercial  and  military  warranties,  provided  a  baseline  that 
could  be  tailored  to  fit  individual  engine  programs  (65). 


Guide  (for  military  aircraft  turbine  engines)  was  published. 
A  Joint  Logistics  Commanders *  ad  hoc  group  with  warranty 
expert  representatives  from  the  Air  Force,  the  Army,  and  the 
Navy  developed  and  co-authored  the  guide  (80:36).  Its 
purpose  was  to  tie  together  the  technical  information  needed 
by  program  managers,  and  to  provide  a  framework  for  creating 
and  applying  engine  warranties  (80:36). 

From  this  brief  history  of  engine  warranties,  it  can  be 
seen  that  the  use  of  Air  Force  engine  warranties  (other  than 
COD)  is  relatively  new.  However,  substantial  interest  and 
involvement  have  made  engine  warranties  for  the  Air  Force 
far  more  than  exploratory  and  has  increased  their  use  on 
engine  contracts. 

Background  Information  on  Acquisition  of 
Air  Force  Engines  and  Warranties 

To  introduce  the  reader  to  specific  Air  Force  processes 
that  pertain  to  engines,  this  section  will  present  some 
general  background  information.  Specifically,  information 
will  be  provided  on  the  engine  acquisition  cycle,  the  timing 
of  warranty  application,  and  relevant  organizational 
structure  and  process  of  obtaining  an  engine  warranty. 

Engine  Acquisition  Cycle .  The  engine  acquisition 


process  conforms  to  the  phases  or  life  cycle  for  Major 
Defense  Systems  Acquisition  found  in  DOD  directive  5000.1. 
The  acquisition  life  cycle  is  essentially  a  logical  flow  of 


activity  representing  on  orderly  progression  from  an 
identification  of  a  system  need  to  final  operational 
deployment  (29).  It  can  be  described  as  a  four  phase 
process.  Included  in  the  acquisition  cycle  are  the  concept 
exploration  phase,  the  demonstration  and  validation  phase, 
the  full  scale  development  phase,  and  the  production  and 
deployment  phase  (29). 

In  the  first  phase,  the  concept  exploration  phase,  the 
need  for  a  new  system  is  identified.  Initial  research  is 
done  to  define  the  system  that  will  best  meet  the  need. 

This  is  followed  by  the  definition  of  the  base  line 
requirements  for  such  a  system  (29). 

The  acquisition  process  then  moves  into  the 
demonstration  and  validation,  phase.  Those  systems  that  have 
met  the  base  line  requirements  are  further  studied.  Within 
this  phase,  system  requirements  are  more  closely  identified 
and  defined.  This  narrows  the  field  to  perhaps  two  or  three 
contractors  whose  designs  meet  the  specifications. 

Prototypes  may  be  developed  to  demonstrate  the  validity  of 
the  particular  design  for  meeting  the  governments ' s  need. 

At  the  end  of  this  phase,  one  design  is  selected  for  full 
scale  development  (29). 

In  the  full  scale  development  phase,  the  selected 
design  is  developed  in  even  more  detail  than  in  the  previous 
phases.  Also,  specifics  for  further  system  development  and 
construction  are  worked  out,  and  the  production  contracts 


are  formalized.  With  the  system  in  detailed  form  and  the 
contract  for  production  finalized,  the  system  then  moves 
into  the  final  phase  (29). 

In  the  production  and  deployment  phase,  the  new  system 
goes  into  production  and  is  distributed  to  the  user.  In 
addition,  throughout  the  phase,  follow-on  production  may  be 
contracted  and  engineering  improvements  may  continue  during 
the  production  and  use  of  the  product  (29). 

In  the  Air  Force,  the  Air  Force  Systems  Command  (AFSC) 
is  responsible  for  the  acquisition  of  major  systems.  Within 
AFSC  are  a  number  of  product  divisions.  One  of  these 
divisions  is  Aeronautical  Systems  Division  (ASD)  which  is 
located  at  Wr ight-Patterson  Air  Force  Base,  Ohio.  ASD 
directs  the  design,  development,  and  acquisition  of  major 
aerospace  systems.  These  include  manned  bombers  and 
fighters,  trainers,  transports,  utility  aircraft,  unmanned 
vehicles,  long-  and  short-range  air-to-surf ace  missiles, 
simulators,  reconnaissance  and  electronic  warfare  equipment, 
and  related  equipment  (10:61-63). 

Within  ASD,  the  Deputy  for  Propulsion  organization  is 
responsible  for  acquiring  the  aircraft  engines,  although  the 
funds  for  the  engines  are  provided  by  the  aircraft  system 
program  office  (SPO)  (26).  For  example,  the  Deputy  for  B-1B 
provided  the  funds  to  the  Deputy  for  Propulsion  to  acquire 
the  F101-GE-102  engines  used  in  the  B-1B  aircraft.  The 
acquisition  cycle  for  both  the  engines  and  the  airframes 
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occurs  simultaneously.  The  aircraft  SPO  establishes  the 
delivery  schedule  for  the  engines  which  are  provided  to  the 
airframe  contractor  as  government  furnished  equipment  (26). 

The  program  management  for  an  engine  changes  during  the 
acquisition  cycle.  Illustrated  in  Figure  3  is  a  generic 
overview  of  the  engine  acquisition  cycle  and  the  responsible 
program  manager?  however,  tracing  the  acquisition  of  a 
specific  engine  could  result  in  a  modification 
(combination/omission)  of  the  acquisition  phases  (26). 

During  the  early  acquisition  phases  (I  and  II  in  Figure 
3)  the  responsible  program  manager  is  in  the  New  Engines  SPO 
(YZN)  under  the  Deputy  for  Propulsion.  Sometime  during  the 
demonstration  and  validation  phase  (II  in  Figure  3)  and 
through  a  portion  of  the  production/deployment  phase  (IV  in 
Figure  3),  a  program  manager  in  a  specific  propulsion  SPO  is 
identified  to  continue  the  acquisition  management.  The 
specific  propulsion  SPO  is  usually  identified  by  the  type  of 
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aircraft  for  which  the  engine  is  being  made.  As  an  example, 
the  program  manager  for  the  B-1B  engines  is  in  the  Strategic 


Engines  SPO.  During  the  deployment  phase  (part  of  IV  in 
Figure  3),  management  responsibility  of  the  engine  is 
transferred  to  an  Air  Logistics  Center  (ALC)  through  Program 
Management  Responsibility  Transfer  (PMRT).  However,  the 
Deputy  for  Propulsion  remains  responsible  for  the  Air 
Force's  procurement  of  engines  (26). 

Applying  Engine  Warranties .  Having  looked  at  the 
engine  acquisition  cycle,  the  next  question  is  when  during 
the  acquisition  cycle  can  the  engine  warranty  be  applied. 
Simply,  an  engine  warranty  can  be  applied  either  early  or 
late  in  the  acquisition  cycle  (48:23).  Each  has  its  own 
advantages  and  drawbacks  as  explained  below. 

An  early  warranty  application  could  occur  during  the 
end  of  the  concept  exploration  phase  in  which  several 
contractors  may  be  bidding  against  each  other  (48:23; 

82:20).  Although  the  competition  would  be  favorable  for  the 
government,  the  drawback  would  be  that  it  is  early  in  the 
program  and  the  engine  specifications  are  not  well  known 
(48:23  82:20,21).  The  result  may  make  actual  contract  price 
negotiations  between  the  Air  Force  and  the  contractor 
difficult  (48:23). 

A  late  warranty  application  could  occur  during  the 
development  and  production  phase.  The  main  advantage  may  be 
that  firm  data  on  engine  failures  and  reliability  would  be 


available  (48:23).  However,  the  drawback  is  that  there  may 
be  only  one  contractor  bidding  which  could  result  in  the 
government  not  being  able  to  receive  the  advantages  of 
competition . 

Ideally,  warranties  are  developed  after  field  failure 
data  has  been  accumulated  from  operational  tests  conducted 
during  the  demonstration  and  validation  phase;  then,  they 
are  implemented  during  the  full  scale  development  or  the 
production  phase  (82:21). 

Obtaining  an  Engine  Warranty .  The  next  procedure  to  be 
examined  is  obtaining  an  engine  warranty.  Specifically,  the 
individuals  involved  and  the  process  of  obtaining  an  engine 
warranty  will  now  be  discussed. 

As  shown  in  Figure  4,  the  sequence  of  events  taken  to 
obtain  an  engine  warranty  involves  the  interaction  of 

INTERNAL 

Deputy  for  Propulsion  Organization 
Program  Manager,  YZ (N/S/A/F/Y) 

Contracting,  YZK 
Logistics,  YZ ( N/S/A/F/Y) L 
Resources  Management,  YZPR 
Engineering,  YZ(N/S/A/F/Y)E 
Plans  and  Requirements,  YZLR 
Deputy  Commander  for  Propulsion,  YZ 


Functional  Disciplines  Involved  to  Obtain  an  Engine  Warranty 

Figure  4 


individuals  from  several  functional  disciplines,  both 
internal  and  external  to  the  Deputy  for  Propulsion 


organization.  The  internal  individuals  include  the  Deputy 
Commander  for  Propulsion,  the  program  manager,  and  personnel 
from  contracting,  logistics,  plans  and  requirements, 
resources  management,  and  engineering;  the  external 
individuals  are  personnel  from  legal  and  pricing  (66). 

Illustrated  in  Table  I,  are  the  steps  in  the  process  of 
obtaining  a  warranty  and  who  is  involved.  As  the 
illustration  is  a  general  overview,  the  timing  and 
occurrence  of  each  step  is  dependent  on  the  engine  program 
(66)  . 

Steps  1  through  8  are  performed  prior  to  negotiations 
in  order  to  establish  the  Air  Force  position  of  the  value  of 
a  warranty  in  relationship  to  the  systems  Life  Cycle  Cost 
( LCC )  . 

The  program  manager  establishes  the  warranty 
objectives.  Then  the  warranty  clause  is  written.  The 
responsible  functional  discipline  for  writing  the  warranty 
clause  is  dependent  on  the  specific  engine  program. 
Typically,  the  responsibility  has  been  from  one  of  the 
following  functions:  plans  and  requirements,  program 
manager,  or  contracting  (66). 

The  warranty  clause  is  reviewed  by  legal,  contracting, 
and  logistics  to  determine  if  it  is  both  in  compliance  with 
the  law  and  is  enforceable.  LCC  estimates  of  the  system  are 
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TABLE  I 


OBTAINING  AN  ENGINE  WARRANTY 


STEPS 


WHO  DOES 


1.  Establish  warranty  objectives  Program  Manager 


2.  Review  and  interpret  warranty 


3.  Obtain  latest  Life  Cycle  Cost 

( LCC )  Data 

4.  Obtain  current  warranty  price 

data,  including  Government 
administration  cost 

5.  Perform  technical  evaluation 

6.  Calculate  warranty  benefits 

7.  Perform  LCC  analysis 


8.  Issue  Request  for  Proposal 

9.  Begin  negotiations 

0.  Conduct  Risk  Assessment 

1.  Perform  reliability/ 

sensitivity  analysis 

2.  Conclude  Negotiations 


Legal /Contracting/ 
Logistics 

Logistics 


Resources  Management,/ 
Contracting 


Engineering 

Plans  and  Requirements 

Plans  and  Requirements/ 
Resources  Management 

Contracting 

Contract ing/ Pricing 

Engineering 

Plans  and  Requirements/ 
Engineering 

Contracting 


3.  Perform  cost-benefit  analysis  Plans  and  Requirements 


4.  Make  cost  effectiveness 

determination 

5.  Place  warranty  on  contract 

or  initiate  waiver  request 


Program  Manager/Deputy 
Commander  for  Propulsion 

Contracting 


obtained  for  both  with  and  without  the  warranty  (steps  3 

through  7 ) . 

During  negotiations  (steps  9  through  12)  additional 
data  that  could  change  the  LCC  estimates  are  considered. 

The  Air  Force  conducts  a  risk  assessment  and  performs 
reliability/sensitivity  analysis.  The  LCC  analysis  is 
updated . 

After  negotiations  (steps  13  through  15)  a  cost-benefit 
analysis  is  completed  and  a  cost  effectiveness  determination 
is  made.  Finally,  if  the  warranty  clause  is  determined  to 
be  cost  effective  it  is  put  on  contract.  On  the  other  hand, 
if  the  warranty  clause  is  not  determined  to  be  cost 
effective  a  request  for  a  warranty  waiver  is  initiated. 

Currently,  there  is  no  official  definition  for  cost 
effectiveness  criteria  (9:31;  23:11-31-11-34).  The  criteria 
currently  used  by  the  Deputy  for  Propulsion  organization  to 
make  the  cost  effectiveness  decision  for  engine  warranties 
is  based  on  the  contractor's  profit  margin  allowed  on  the 
contract.  The  warranty  is  considered  cost  effective  if  the 
profit  margin  for  the  warranty  is  equal  to  or  less  than  the 
profit  margin  allowed  on  other  parts  of  the  contract  (66). 

For  example,  assume  (1)  that  the  profit  margin  for  the 
contract  is  15%;  and  (2)  that  the  government's  costs  minus 
benefits  in  the  cost-benefit  analysis  is  considered  to  be 
the  contractor's  warranty  profit.  Then,  the  warranty  is 
considered  cost  effective  if  the  estimated  warranty  profit 


to  the  contractor  is  less  than  or  equal  to  15%  of  the 
government's  costs  for  the  warranty. 


The  warranty  clause  put  on  contract  not  only  specifies 
the  warranted  conditions,  but  could  also  request  that  the 
contractor  is  to  provide  cost  and  reliability  data  to  the 
government.  The  cost  and  reliability  requests  are  put  on 
contract  by  using  the  contract  data  requirements  list  (CDRL) 
(66)  . 

The  preceding  background  information  briefly  introduced 
the  reader  to  some  of  the  procedures  of  acquiring  engines 
and  engine  warranties.  In  addition,  some  differences  that 
can  occur  were  highlighted.  For  example,  during  the 
engine's  acquisition  cycle  the  program  management  changes 
and  engine  warranties  can  be  applied  at  different  times. 
Also,  many  individuals  are  involved  with  obtaining  an  engine 
warranty,  and  the  timing  and  occurrence  of  the  steps  in  the 
process  can  be  engine  program  specific.  The  next  section 
will  identify  the  specific  engines  to  be  studied  in  this 
research. 

Identifying  the  Specific  Engines  to  be  Studied 

In  order  to  determine  which  engines  to  study,  a  list  of 
jet  engines  currently  in  the  Air  Force  Inventory  was 
compiled,  see  Table  II.  The  list  shows  the  model 
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JET  ENGINES  CURRENTLY  IN  THE  AIR  FORCE  INVENTORY 


Engine  Model 
Designation 

Aircraft  Using 

Engine 

Engine 

Manufacture 

Warranty 

F100-PW-100 

F-15A/B/D;  YF-15A 

PW 

YES 

F100-PW-200 

F-16A/B/C/D 

PW 

YES 

F101-GE-102 

B-1B 

GE 

YES 

F108-CF-100 

KC-135R 

GE 

YES 

F109-100 

T46A 

GARRETT 

YES 

F100-PW-220 

F-15C/D/E 

PW 

YES 

PW  2037 

Proposed  C-17 

PW 

YES 

F110-GE-100 

F-16D/C 

GE 

YES 

TF34-GE-100 

A-10A/B 

GE 

YES 

J85-GE-5 

T-38A 

GE 

NO 

J85-GE-5L 

AT-38B;  T-38A 

GE 

NO 

J85-GE-1 3C/-1 3D 

F-5B 

GE 

NO 

J85-GE-1 7 ( POD ) /- 

■17A  C-123K;  A/OA-37B 

GE 

NO 

TF39-GE-1A/-1C 

C-5A/B 

GE 

NO 

J85-GE-2IA/-21B 

F-5E/F 

GE 

NO 

J69-T-25/-25A 

T-37B 

TELEDYNE 

NO 

T76-G-10A/-418 

OV-10A  LEFT 

GARRETT 

NO 

T76-G-10A/-419 

OV-lOA  RIGHT 

GARRETT 

NO 

J57-P-19W/-29WA 

NB-52B ;  B-52D 

PW 

NO 

J57-P-43WB 

C/NC/NKC-135A;  EC-135H;  PW 

B-52G 

NO 

J57-P-59W 

C/EC/KC/NKC-135A; 
EC-135D/Q;  EC-135G/L 

PW 

NO 

J79-GE-1 5 

F/RF-4C;  F-4D 

GE 

NO 

J79-GE-1 7 

F-4E/G 

GE 

NO 

J75-P-17 

F-106A/B 

PW 

NO 

J75-P-19W 

F-105D/F 

PW 

NO 

NOTE:  PW  =  PRATT  WHITNEY 

GE  =  GENERAL  ELECTRIC 


Continued  next  page 
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JET  ENGINES  CURRENTLY  IN  THE  AIR  FORCE  INVENTORY 


1 


Engine  Model 

Aircraft  Using 

Engine 

Designation 

Engine 

Manufacture 

Warranty 

TF33-P-3 

B-52H 

PW 

NO 

TF33-P-5 

C/EC/WC-135B;  C-135C 
RC-135M/S/W 

* 

PW 

NO 

TF33-P-7/-7A 

C-141A/B;  NC-141A 

PW 

NO 

TF33-P-9 

EC-135C/J;  RC-135U/V 

PW 

NO 

TF33-PW-100 

E-3A 

PW 

NO 

TF33-PW-102 

C/EC/KC/NKC-135E; 
EC-135H/K/P?  RC-135T 

PW 

NO 

T33-PW-102A 

C-18A 

PW 

NO 

TF41-A-1A/-1B 

A/AY-7D?  A-7K 

ALLISON 

NO 

TF30-P-3 

F/EF-111A;  F-111F 

PW 

NO 

TF30-P-7 

FB-111A 

PW 

NO 

TF30-P-9 

F-111D 

PW 

NO 

TF30-P-100 

F-111F 

PW 

NO 

T56-A-7B 

C-130B;  C/EC/WC-130E 

ALLISON 

NO 

T56-A-9B/-9C/-9D 

AC/C/DC-130A;  C-130D 

ALLISON 

NO 

T56-A-15 

MC-130E ,*  HC-130N/P; 

ALLISON 

NO 

AC/C/DC/EC/HC/WC-1 30H 

J33-A-35/35A 

T-33A 

ALLISON 

NO 

F103— GE-100 

E-4A/B;  KC-10A 

ALLISON 

NO 

JT3D-3B 

VC-137B/C 

PW 

NO 

JT8D-9 

C-9A;  T-43A 

PW 

NO 

J58 

SR-71A 

PW 

NO 

J60-P-3/-3A 

CT/NT/T-39A;  T-39B 

PW 

NO 

J60-P-5A/-5B 

tr-ia/b 

PW 

NO 

PT6A-20 

C-6A 

PW 

NO 

PT6A-27 

UV-18A 

PW 

NO 

NOTE: 

PW 

=  PRATT  WHITNEY 

GE 

=  GENERAL 

ELECTRIC 

The  criteria  used  to  select  the  engines  for  this 
research  was  that  the  engines  have  an  expired  warranty. 

This  criteria  was  chosen  in  order  to  make  sure  all  the  costs 
would  have  been  incurred  for  the  assessment  of  the  cost 
effectiveness . 

The  selection  criteria  narrowed  the  choices  to  some  of 
the  F100-PW-100  and  F100-PW-200,  and  the  TF34-GE-100 
engines.  The  F100— PW-100  engines  are  used  to  power  the  twin 
engine  McDonnell  Douglas  F-15  Eagle  fighter  aircraft,  and 
the  F100-PW-200  engines  power  the  single  engine  General 
Dynamics  F-16  aircraft;  these  engines  were  produced  by  Pratt 
and  Whitney  Group  of  United  Technologies  Corporation 
(51:885).  The  TF34-GE-100  engines  are  used  to  power  the 
twin-engined  Fairchild  Republic  A-10A  Thunderbolt  II;  they 
were  produced  by  General  Electric  Company  Aircraft  Engine 
Business  Group  (51:870). 

From  these  engines,  a  judgmental  selection  was  made 
based  on  how  long  the  warranty  has  been  in  effect,  the 
number  of  claims  processed,  and  the  limited  time  to  conduct 
this  research.  The  engines  selected  were  the  F100-PW-100 
and  the  F100-PW-200  which  are  no  longer  under  warranty; 
specifically.  Lot  IX  engines.  [A  "Lot"  is  an  annual 
purchase  of  engines.]  Hereafter  the  F100-PW-100  and  the 
F100-PW-200  will  be  simply  refered  to  as  the  F100  engines. 

Table  III  provides  a  description  of  the  warranties  used 
for  the  F100  engines. 
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TABLE  III 


F100  ENGINE  WARRANTY  DESCRIPTIONS 


t 


The  contractor. 


Supply  Warranty 

Pratt  and  Whitney,  warranted  that  they 


i  would  repair  or  replace  any  part  found  to  be  defective  in 

material  or  workmanship  within  240  days  of  first 
installation  in  an  aircraft  when  installed  within  one  year 
after  acceptance  date. 

High  Pressure  Turbine  Warranty 
The  contractor  warranted  that  they  would  repair  or 
replace  high  pressure  turbines  found  to  be  unserviceable  to 
the  limits  of  the  technical  orders  for  durability  reasons 
within  900  equivalent  Tactical  Air  Command  (TAC)  cycles  or 
42  months  from  acceptance  date  whichever  occurred  first. 

Expanded  Engine  Warranty 

The  contractor  warranted  that  they  would  repair  or 
replace  any  engine,  module,  or  accessory  found  to  be 
unserviceable  to  the  limits  of  the  technical  orders  within 
the  first  200  hours  total  operating  hours  (TOT)  or  two  years 
from  acceptance  date,  whichever  occurred  first.  Also,  this 
warranty  covered  secondary  deimage  to  the  warranted  engine. 

Fan  Disk  Warranty 

The  contractor  warranted  the  fan  disk  against  any  disk 
failure  caused  by  a  manufacturing  flaw  for  a  period  of  3000 
equivalent  TAC  cycles  or  ten  years  after  original  acceptance 


the  additional  pipeline  costs  in  shipping  and  repair  times 
which  would  result  in  meeting  the  warranty  requirement,  that 
all  repairs  were  to  be  made  at  the  contractor's  (Pratt  and 
Whitney)  facilities  rather  than  the  government  depot.  A 
determination  was  made  to  accept  the  coverage  in  view  of 
substantial  positive  benefits  (81:17,18). 

Both  primary  and  secondary  damage  was  covered;  a 
"first"  in  military  coverage  for  a  fighter  jet  engine.  This 
gave  the  military  warranty  coverage  similar  to  the  coverage 
that  had  been  offered  to  the  commercial  aircraft  industry 
for  years  ( 38 ) . 

The  High  Pressure  Turbine  Warranty  was  followed  with 
the  addition  of  the  Expanded  Engine  Warranty.  This  Lot  IX 
warranty  was  developed  and  applied  due,  in  a  large  part,  to 
the  emphasis  Headquarters  Air  Force  Systems  Command  was 
placing  on  the  desire  for  more  meaningful  warranties. 

Before  this  warranty  was  applied,  the  government  was  eight 
years  and  1845  engines  into  the  F100  program  (38). 

Beginning  with  Lot  X  F100  engines,  the  High  Pressure 
Turbine  Warranty  was  expanded  to  include  coverage  for  1350 
Tactical  Air  Command  (TAC)  cycles  [a  cycle  is  a  unit  of 
measure  for  engines]  or  five  years  from  the  acceptance  date. 
Also,  beginning  with  the  Lot  X  engines,  all  fan  disks 
supplied  in  the  engine  and  modules  were  warranted  for  a 
period  of  3000  TAC  cycles  or  ten  years  after  the  acceptance 
date  (23:111-8). 
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In  this  chapter  a  brief  history  of  Air  Force  engine 
warranties  was  given.  Background  information  on  selected 
Air  Force  procedures  pertaining  to  engines  and  engine 
warranties  was  provided.  Finally,  a  selection  of  the 
engines  to  be  used  in  this  study  was  made.  The  next  chapter 
will  discuss  the  feasibility  of  obtaining  the  information 
required  for  assessing  the  benefits  and  costs  of  jet  engine 
warranties  using  the  approach  described  in  chapter  two. 


i 

i 


IV.  Evaluating  and  Measuring  Engine 
Warranty  Costs  and  Benefits 

The  purpose  of  this  chapter  is  to  determine  the 
feasibility  of  obtaining  the  information  required  for 
assessing  the  benefits  and  costs  of  jet  engine  warranties. 

In  order  to  do  this,  the  nature  and  availability  of  the 
information  needed  to  analyze  DOD  warranties  is  described. 
Specifically,  the  information  required  to  evaluate  and 
measure  warranty  costs  and  benefits  is  discussed  along  with 
examples  of  data  obtained  from  engine  warranties.  The  first 
section  considers  the  information  required  to  evaluate 
warranty  costs.  Next,  the  problems  involved  in  measuring 
warranty  costs  are  considered  and  some  of  the  problems  found 
in  obtaining  these  costs  are  discussed.  Finally,  the 
problems  involved  in  evaluating  and  measuring  warranty 
benefits  are  approached  in  the  same  manner. 

Evaluating  Warranty  Costs 

In  chapter  two  the  information  required  to  evaluate  the 
cost  of  DOD  warranties  was  identified.  The  costs  to  DOD 
were  classified  as  explicit  and  implicit.  The  explicit  cost 
of  warranties  was  defined  as  the  manufacturer's  total  costs 


for  a  given  reliability  level  plus  the  manufacturer's 
profit.  The  implicit  cost  of  warranties  was  defined  as 
DOD's  costs  necessary  to  enforce  the  warranty. 


The  model  in  chapter  two,  illustrated  again  in  Figure 
6,  describes  the  manufacturer's  total  cost  as  a  function  of 


the  manufacturing  cost  and  the  warranty  claim  cost.  The 
manufacturer  can  make  a  trade-off  between  these  two  costs. 

In  addition,  the  manufacturer,  in  order  to  maximize  profit 
(when  revenue  is  assumed  to  be  fixed),  strives  for  the  cost 
which  corresponds  to  the  minimum  point  on  the  total  cost 
curve . 

As  explained  in  chapter  two,  in  order  to  make  the  best 
decisions  regarding  warranties,  it  is  necessary  know  the 
costs  to  the  manufacturer  and  the  costs  to  DOD.  With  regard 
to  the  manufacturer,  relevant  costs  are  the  actual  cost  (to 
the  manufacturer)  of  a  failure,  the  number  of  failures  which 
occurred  at  a  given  level  of  warranty  coverage,  and  the 
manufacturer's  cost  of  building  reliability  into  the 
product.  From  a  DOD  perspective,  it  is  necessary  to  know 
the  price  paid  for  the  warranty  (the  explicit  cost)  and 
also,  the  internal  cost  of  documenting  and  complying  with 
the  warranty  claim  provisions  (the  implicit  cost). 

Interpretation  of  Terms .  Before  expounding  on 
measuring  the  costs  to  the  manufacturer  and  the  costs  to 
DOD,  it  will  be  useful  to  interpret  the  terms  "failure"  and 
"reliability"  as  used  in  the  model  (Figure  6) . 

Defining  "Failure" .  "Failure"  is  a  generic 
term  used  in  the  model  to  describe  a  condition  that  the 
manufacturer  is  responsible  for  correcting.  When  discussing 
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warranties,  "failure"  can  take  on  a  number  of  meanings.  As 
an  example,  it  could  mean  a  complete  breakdown  in  the 
operation  of  an  item,  or  it  could  mean  a  partial  breakdown 
of  the  item.  For  a  partial  breakdown,  the  item  may  still 
operate,  but  not  at  100%  capability. 

Unless  "failure"  is  explicitly  defined,  there  could  be 
problems  in  determining  who  is  responsible  for  fixing  the 
breakdown.  For  the  user,  any  breakdown  of  the  item  should 
be  fixed  by  the  manufacturer.  On  the  other  hand,  for  the 
manufacturer,  any  correction  would  cost  money  and  decrease 
profits.  Therefore,  the  manufacturer  may  be  reluctant  to 
fix  the  item. 

A  warranty  clause  can  alleviate  the  ambiguity  of  the 
term  "failure"  by  describing  the  condition  under  which  a 
claim  can  be  made.  Also,  the  warranty  clause  can  make  the 
responsibility  for  fixing  the  item  legally  binding.  Engine 
warranty  clauses  provide  an  illustration  of  the  diversity  of 
the  term  "failure." 

A  jet  engine  is  a  complicated  piece  of  hardware  with 
3500  plus  parts  and  a  great  number  of  ways  it  could  fail,  or 
not  function  as  specified.  The  engine  "failures"  that  are 
warranted  (to  be  corrected  by  the  contractor),  are 
explicitly  described  in  the  contracts  between  the  Air  Force 
and  the  contractor.  Two  definitions  of  warranted  engine 


failures  follow. 


The  first  example  is  from  the  F100  Lot  IX  engine 
contract.  One  of  the  warranties  in  that  contract  was  the 


Expanded  Engine  Warranty.  A  failure  under  the  terms  of  this 
warranty  was  described  as  the  condition  when  an  engine, 
module,  or  specified  component  was  not  serviceable  in 
accordance  with  procedures  specified  in  applicable  technical 
orders  (4:Atch  8). 

The  second  example  is  from  the  F101-GE-102  Lot  II 
engine  contract.  One  of  the  warranties  in  that  contract  was 
the  Unscheduled  Engine  Removal  and  Selected  Controls  and 
Accessories  (C&A)  Removal  Warranty.  A  failure  under  the 
terms  of  this  warranty  was  described  as  the  condition  when 
the  fleetwide  unscheduled  engine  removal  rate  per  1000 
engine  flying  hours  was  greater  than  specified.  In 
addition,  another  type  of  failure  occurred  if  the  combined 
unscheduled  removal  rate  of  C&A  exceeded  specified 
parameters  (5:11). 

Defining  "Reliability" .  "Reliability"  is  a 
generic  term  used  in  the  model  (Figure  6)  to  describe  the 
quality  of  the  product.  It  is  defined  in  the  Compendium  of 
Authenticated  Systems  and  Logistics  Terms ,  Definitions  and 
Acronyms ,  as  "the  probability  that  a  system,  subsystem,  or 
equipment  will  perform  a  required  function  under  specified 
conditions  without  a  failure,  for  a  specified  period  of  time 
(59:576) . 
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There  are  three  variables  in  the  term  reliability: 
"specified  conditions,"  "failures,"  and  "time."  These  terms 
must  be  strictly  defined  when  applying  any  reliability 
parameter  to  a  warranty.  When  discussing  warranties, 
"reliability"  can  take  on  different  meanings.  Therefore,  it 
is  important,  when  measuring  the  costs  of  warranties,  that 
the  term  reliability  is  defined  for  the  specific  warranty 
being  evaluated.  Engine  warranties  can  provide  an 
illustration  of  the  differences  in  meanings.  Two  examples 
are  provided  below. 

The  first  example  is  from  the  F100-PW-220  engine 
contract.  The  Warranty  Part  1  describes  one  engine 
performance  parameter.  The  contractor  warrants  that  the 
performance  of  each  engine  delivered,  for  the  period 
specified,  shall  not  be  less  than  98%  of  the  intermediate 
thrust  as  set  forth  in  specification  and  shall  not  exceed 
105%  of  the  intermediate  specific  fuel  consumption  as  set 
forth  in  the  specification  (6:205). 

The  second  example  is  from  the  F100  Lot  IX  engine 
contract.  The  Warranty  of  Supply  describes  the  quality 
parameter  of  the  product.  The  contractor  warrants  that  at 
the  time  of  delivery  all  supplies  f  rnished  under  this 
contract  shall  be  free  from  defects  in  material  and 
workmanship  ar.d  shall  conform  with  the  purchase  description 
and  all  other  requirements  of  this  contract  (3:35). 
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From  the  interpretation  of  the  terms  "failure"  and 
"reliability"  used  in  the  model  (Figure  6),  it  can  be  seen 
that  they  can  represent  many  different  ideas.  In  addition, 
when  they  are  used  in  discussing  warranties,  each  term  must 
be  explicitly  defined. 

Measuring  Warranty  Costs 

To  measure  warranty  cost,  DOD  must  obtain  or  estimate 
the  costs  to  the  manufacturer  and  the  costs  to  DOD. 

Manufacturer ' s  Costs .  For  the  manufacturer,  three 
elements  of  information  are  required.  One,  the  actual  cost 
of  a  failure.  Two,  the  number  of  failures.  Three,  the  cost 
of  building  reliability  into  the  product. 

Actual  Cost  of  a  Failure .  The  first  element  of 
information  to  measure  costs  to  the  manufacturer  is  the 
actual  cost  of  a  failure. 

What  is  Required .  The  cost  of  a  failure 
depends  on  the  contract  warranty  clause  and  what  the 
contractor's  obligations  are.  The  contractor's  obligations 
can  include  the  costs  to  remedy  or  take  corrective  action 
when  a  warranted  failure  occurs.  Costs  are  also  incurred  by 
providing  a  warranty  on  the  item.  The  cost  include  the  cost 
of  preparing  reports  to  be  submitted  to  the  government,  and 
also,  the  cost  of  a  program  in  order  to  administer  the 
warranty  claims  (13:4-13;  46:117).  An  explanation  of  the 
manufacturer's  remedy,  report,  and  administration  costs 


follows . 


Remedies .  The  way  in  which  the 
contractor  will  correct  the  failure  is  called  the  remedy  and 
is  specified  in  the  contract.  According  to  the  Defense 
Federal  Acquisition  Regulation  Supplement,  there  are  three 
basic  remedies  the  contractor  can  take  (13:D-4).  One,  the 
contractor  takes  such  action  as  repair,  replace,  and/or 
redesign  at  no  additional  cost  to  the  government.  Two,  the 
contractor  is  responsible  for  reimbursing  the  government  for 
actions  taken  to  correct  the  failure.  Three,  the  contractor 
provides  an  equitable  adjustment,  such  as  a  reduction  in 
contract  price,  in  the  event  of  a  failure. 

Engine  warranties  have  been  written  in  which  the 
contractor  was  required  to  take  corrective  action  of  a 
failure.  One  example  is  in  the  F100  Lot  IX  High  Pressure 
Turbine  Warranty.  In  addition,  engine  warranties  have  been 
written  in  which  the  government  would  receive  an  equitable 
adjustment  in  the  contract  price  when  a  failure  was  repaired 
by  the  government.  One  example  is  in  the  F101-GE-102  Lot  II 
Warranty  Part  I . 

Reports .  The  contractor  is  sometimes 
required  to  provide  reports  pertaining  to  the  warranty 
corrections.  These  reports  can  be  used  by  the  government  to 
implement  certain  warranted  turnaround  times,  to  maintain 
appropriate  inventory  and  configuration  control,  and  to 
assess  the  effectiveness  of  the  warranty  (13:4-13). 


For  engines,  warranty  reports  required  from  the 
contractor  are  identified  in  the  warranty  clause  of  the 
contract.  An  example  of  one  type  of  report  is  a  monthly 


warranty  performance  report.  With  the  report,  the 
contractor  provides  data  of  the  action  taken  under  the 
warranty  including  the  cost  of  the  action.  This  type  of 
report  was  required  by  the  contractor  for  the  F100  Lot  IX 
Expanded  Engine  Warranty.  The  monthly  report,  Contractor 
Warranty  Cost  Report,  included  the  item  nomenclature,  the 
serial  or  part  number,  the  total  operating  time  [the 
reliability  measurement  being  warranted],  the  reported 
problem,  the  number  of  days  to  satisfy  claim  [turnaround 
time],  and  the  costs  incurred,  both  during  the  month  and 
upto  the  reporting  date. 

Contractor  Warranty  Administration.  A 
contractor's  program  to  administer  the  warranty  claims  can 
involve  office  personnel,  engineering  and  manufacturing 
functions,  and  also,  establishing  an  area  for  working  on 
failures  and/or  storing  replacement  parts  for  anticipated 
failures  (46:117).  An  example  of  a  program  to  administer 
engine  warranty  claims  follows. 

For  one  engine  contractor,  there  is  an  organizational 
group  which  is  responsible  for  the  overall  management, 
repair  disposition,  vendor  direction,  coordination  with  the 
Air  Force,  and  reporting  on  the  warranty  program  efforts. 

In  addition,  warranted  item  repair  facilities  are  located  at 
the  contractor  and  some  of  its  vendor  plants  (35:32). 


Obtaining  the  Cost  of  a  Fai lure .  For  DOD  to 
obtain  the  manufacturer's  cost  of  a  failure,  two  possible 
scenario's  exist  with  regard  to  timing.  The  first  involves 
obtaining  the  information  prior  to  the  warranty  in  order  to 
perform  the  cost-benefit  analysis  and  pricing  the  warranty. 
The  second  scenario  involves  obtaining  the  information  after 
the  warranty  expires  in  order  to  evaluate  the  warranty 
effectiveness . 

Prior  to  the  Warranty .  Prior  to  the 
warranty,  DOD  estimates  the  manufacturer's  cost  of  a  failure 
based  on  technical  analysis  of  projected  failures  and  the 
contractor's  remedy  cost  for  those  failures  (7;  51:7-4,7-5; 
66).  Also,  the  manufacturer's  cost  could  be  provided  by  the 
contractor  during  its  proposal  submission  (66;  69).  To 
acquire  a  warranty  with  an  item,  the  government  obtains  the 
contractor's  proposal  of  the  warranty  price.  Under  the 
disclosure  requirements  of  Public  Law  87-653,  Truth  in 
Negotiations  Act,  the  contractor  is  responsible  for 
substantiating  the  proposal  with  current,  accurate,  and 
complete  cost  and  pricing  data  (46:87;  67:41;  69).  The 
proposal  is  subject  to  audit  by  the  Defense  Contractor  Audit 
Agency.  For  some  contractors,  because  warranties  have  only 
recently  become  mandatory,  there  is  lack  of  a  historical 
warranty  data  base  to  support  the  proposal  (74).  However, 
for  engine  contractors,  this  is  not  true  (69).  For  example, 
one  engine  contractor  maintains  a  warranty  cost  accumulation 
system  that  has  been  operational  for  over  25  years  (59:31). 


After  the  Warranty  Expires .  After  the 
warranty  expires,  if  there  was  a  requirement  in  the  contract 
for  the  contractor  to  submit  information  for  the  cost  of  a 
failure,  then  it  could  be  used  for  evaluating  the  warranty. 

This  was  the  case  for  the  F100  Lot  IX  Expanded  Engine 
Warranty.  The  report  titled.  Contractor  Warranty  Cost 
Report,  provided  the  cost  incurred  for  the  reporting  period 
and  total  expenditures  as  of  the  reporting  date.  The  report 
listed  the  warranty  cost  for  the  F100-PW-100  engines,  that 
power  the  F-15  aircraft,  separately  from  the  warranty  cost 
for  the  F100-PW-200  engines,  that  power  the  F-16  aircraft. 

In  addition,  the  cost  were  divided  into  three  categories: 
engine/module;  components;  and  tooling.  The  total 
contractor's  costs,  shown  in  Table  IV,  was  $4,914,000. 


TABLE  IV 

Total  Contractor  Warranty  Cost 
for  F100  Engine  Lot  IX 

from  7) 


Engine  Type 

Engine /Modules 

Components 

Tooling 


(Adapted 

F100-PW-100 

723,000 

3,071,000 

985,000 

$4,779,000 


F100-PW-200 


135,000 


Totals 


723,000 

3,206,000 

985,000 


Total 


$  135,000 


$4,914,000 


Also,  Table  IV  shows  that  the  distribution  of  the  cost  was 
$4,779,000  for  the  F100-PW-100  engines  and  $135,000  for  the 
F100-PW-200  engines. 

After  the  warranty  expires,  if  there  was  not  a  contract 
requirement  for  the  contractor  to  submit  information  for  the 
cost  of  a  failure,  it  would  be  difficult  to  obtain  the 
information. 

One  suggested  method  to  obtain  the  data  is  to  use  Air 
Force  maintenance  records.  To  first,  determine  the  failures 
of  the  units  covered  by  the  warranty.  Secondly,  to  estimate 
the  costs  of  the  failure  by  using  labor/material  cost  and/or 
the  cost  of  replacement  parts.  Finally,  use  the  estimates 
as  the  cost  of  the  warranty  claims  in  evaluating  the 
warranty.  Two  problems  occur  with  this  method.  The  first 
problem,  is  that  the  maintenance  data  may  not  be  accurate 
due  to  inherent  errors  in  military  data  collection  systems 
(13:4-4;  61;  79).  One  cause  for  the  errors  is  that  no  entry 
or  an  incorrect  entry  is  made  when  a  failure  occurs,  because 
the  maintenance  worker,  who  is  responsible  for  making  the 
entry,  has  no  vested  interest  in  the  accuracy  or  the 
usefulness  of  the  data  (61).  The  second  problem  with  using 
the  maintenance  records  is  that  the  other  contractor  claim 
costs,  such  as  preparing  reports,  administering  the 
warranty,  and  operating  repair  facilities,  are  not 
considered.  This  would  result  in  an  underestimated  warranty 
claim  cost  for  the  contractor. 


Number  of  Failures ♦  The  second  element  of 
information  required  to  measure  the  cost  to  the 
manufacturer,  in  order  to  evaluate  a  warranty,  is  the  number 
of  failures.  As  previously  noted,  a  failure  is  a  warranted 
condition  that  the  contractor  is  required  to  remedy.  Also, 
the  term  failure  must  be  explicitly  defined  in  the  warranty 
clause . 

What  is  Required .  The  number  of  failures  is 
needed  in  order  to  evaluate  the  warranty  cost.  According  to 
the  1986  Government  Contract  Costs  Manual ,  the  contractor 
and  the  government  must  have  some  idea  of  the  number  of 
failures  in  order  to  price  a  warranty  (46:117).  The 
difficulty  in  estimating  the  expected  number  of  failures 
contributes  to  the  risk  of  the  warranty  (13:3-14).  For 
example,  take  the  case  of  a  new  item  being  produced.  The 
new  item  has  no  historical  data  to  use  for  estimating  the 
expected  number  of  failures.  The  risk  to  the  contractor,  is 
the  potential  liability  that  may  be  incurred  if  the  number 
of  failures  is  underestimated.  The  risk  to  the  government 
if  the  number  of  failures  is  overestimated,  is  the  potential 
that  the  warranty  claim  cost  (a  function  of  the  number  of 
failures  and  the  cost  of  the  failures)  will  not  assure  a 
reliable  product  is  delivered  and/or  that  the  government 
will  pay  too  much  for  the  warranty. 

On  the  other  hand,  take  the  case  of  the  item  that  is  in 
production.  The  historical  failure  data  on  previous  lots 
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provide  information  to  both  the  government  and  the 
contractor  (13:3-14).  This  data  can  be  used  to  better 
estimate  the  expected  number  of  failures.  Thus,  reducing 
the  risk  to  both  parties. 

Obtaining  the  Number  of  Failures . 

Prior  to  the  Warranty .  Prior  to  the 
warranty,  the  number  of  failures  must  be  estimated.  The 
estimates  are  based  on  historical  failure  data,  the  results 
of  operational  testing,  and  technical  evaluations  by  an 
engineering  specialist.  If  no  historical  failure  data  is 
available,  data  from  a  like  system  is  used,  with  appropriate 
adjustments  made  for  the  differences  (61;  69;  71). 

For  engines  there  is  an  enormous  amount  of  historical 
data  available  in  the  maintenance  record  systems  maintained 
by  Headquarters  Air  Force  Logistics  Command  (57).  However, 
problems  occur  when  a  performance  measurement,  or  parameter, 
being  warranted  is  not  captured  in  the  maintenance  records 
(20;  27).  As  an  example,  "mean  time  between  failure"  (MTBF) 
is  a  contractual  measurement  for  reliability.  However, 
there  is  no  historical  maintenance  data  on  the  MTBF  of  an 
item.  The  maintenance  records  have  data  for  when  the  item 
was  serviced.  All  engine  service  can  be  considered  either 
preventive  maintenance  or  corrective  maintenance. 

Preventive  maintenance  is  performed  so  that  a  failure  will 
not  occur,  while  corrective  maintenance  takes  place  after  a 


failure  has  occurred  (37:7).  These  entries  can  be  used  to 


obtain  a  mean  time  between  maintenance"  ( MTBM ) . 

Maintenance  action  on  an  item  does  not  equate  to  a  failure 
of  the  item,  therefore,  MTBM  does  not  equal  MTBF  (20). 

After  the  Warranty  Expires .  After  the 
warranty  expires,  if  there  was  a  requirement  in  the  contract 
for  the  contractor  to  submit  information  for  the  number  of 
failures,  then  it  could  be  used  for  evaluating  the  warranty. 

This  was  the  case  for  the  F100  Lot  IX  Expanded  Engine 
Warranty.  A  monthly  report  titled,  Contractor  Warranty  Cost 
Report,  accounted  for  the  number  of  failures  returned  to  and 
processed  by  the  contractor.  Two  of  the  twenty-nine  types 
of  components  covered  under  the  Expanded  Engine  Warranty  for 
the  F100  Lot  IX  engines  were  the  unified  fuel  controls  (UFC) 
and  the  engine  electronic  controls  (EEC).  Figure  7  shows 
the  projected  number  of  failures  and  the  number  of  claimed 
failures  for  these  two  types  of  components.  In  can  be  seen 
in  Figure  7,  that  in  both  cases,  the  number  of  claimed 
failures  were  about  30  percent  of  the  projected  number  of 
failures . 

One  reason  for  the  discrepancy  was  that  some  of  the 
failures  under  this  warranty  were  not  claimed,  instead 
repairs  were  made  by  the  Air  Force  which  voided  the  warranty 
(23:IV-11;  39:17).  The  reason  for  the  Air  Force  repairs  was 
attributable  to  problems  in  warranty  implementation,  which 
included  weak  initial  warranty  claim  procedures,  inadequate 
field  level  training,  and  problems  in  screening  procedures 
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at  the  depot  to  prevent  voiding  warranties  (23:IV-9,  IV-10; 
33;  55). 

Warranted  failures  that  should  be  corrected  by  the 
contractor  but  are  corrected  by  the  Air  Force  voids  the 
warranty  and,  results  in  two  conditions.  The  first 
condition  is  a  decrease  in  the  warranty  claim  cost  (a 
function  of  the  the  number  of  failures  and  the  cost  of  a 
failure)  for  the  contractor.  The  decrease  could  be  enough 
so  that  the  warranty  claim  cost  is  not  a  factor  in  assuring 
a  reliable  product.  Also,  the  contractor,  by  not  paying  for 
unclaimed  failures,  increases  its  profit. 

The  second  condition  of  the  voided  warranty  is  the 
additional  internal  cost  to  DOD.  When  the  repairs  should  be 
made  by  the  contractor  but  are  made  by  the  Air  Force,  then 
the  costs  of  the  repairs  are  additionally  incurred  costs 
which  could  have  been  avoided.  This  condition  is  discussed 
in  the  section  of  DOD's  implicit  costs. 

When  determining  the  number  of  claims,  in  order  to 
evaluate  the  warranty  cost,  the  contractor's  monthly  report 
only  provided  a  portion  of  the  data  required.  It  only 
provided  information  on  the  failures  that  were  claimed  by 
the  Air  Force. 

Also  shown  in  Figure  7  are  triangles  that  represent  the 
actual  number  of  UFC  and  EEC  failures  that  occurred  during 
the  warranted  period,  including  both  claimed  failures  and 
voided  warranted  failures.  Only  about  62  percent  of  the 
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projected  unified  fuel  control  failures  actually  failed. 
Also,  only  about  37  percent  of  the  projected  engine 
electronic  control  failures  actually  failed  (7;  39:14-16). 
The  reasons  for  the  difference  between  the  Air  Forces ' s 
projected  number  of  failures  and  the  actual  number  of 
failures  included,  the  Air  Force's  inaccurate  forecast  of 
projected  failures,  and  also,  improvements  in  the  production 
quality  of  the  delivered  engines  (38). 

After  the  warranty  expires,  if  there  was  not  a  contract 
requirement  for  the  contractor  to  provide  information  about 
the  number  of  failures,  or,  as  above,  the  report  provided  by 
the  contractor  is  only  for  a  portion  of  the  warranted 
failures,  a  method,  to  obtain  the  number  of  failures,  would 
be  to  use  the  data  from  Air  Force  maintenance  records. 
However  as  previously  discussed,  to  use  this  method,  the 
warranted  measurement  parameter  must  be  recorded  in  the  data 
base.  In  addition,  the  results  of  using  this  method  may  not 
be  adequate,  because  the  maintenance  data  may  not  be 


completely  accurate  (13:4-4;  61;  79).  As  an  example, 
expressed  in  a  discussion  with  a  specialist  who  was  in  the 
failure  reporting  process,  some  failures  were  exempt  from 
being  reported  (79).  These  failures  were  considered  random 
and  of  a  known  (acceptable)  type.  However,  not  reporting 
the  failures  leds  to  inaccuracies  in  the  data  base. 

The  Cost  of  Building  Reliability  into  the  Product . 
The  third  element  of  information  required  to  measure  cost  to 
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the  manufacturer,  in  order  to  evaluate  warranty  costs,  is 
the  manufacturer's  cost  of  building  reliability  into  the 
product.  As  previously  discussed,  reliability  is  the 
quality  of  the  product.  Also,  reliability  must  be 
explicitly  defined  in  the  warranty  clause. 

What  is  Required.  The  process  of  determining 
and  designing  a  given  reliability  level  into  a  product  is 
the  application  of  scientific  and  engineering  efforts 
(17:6,192;  71).  The  process  begins  in  the  early  phases  of 
the  products  acquisition  cycle.  System  specifications  are 
the  output  of  the  scientific  and  engineering  efforts 
(17:193).  As  the  product  enters  the  production  phase, 
changes  in  reliability  are  accomplished  by  additional 
processes  such  as  quality  control,  changes  in  the  production 
techniques,  and/or  redesign  of  the  product  (17:282,284). 

If  the  system  specifications  produce  the  required 
reliability  level,  the  manufacturing  cost  to  build 
reliability  into  a  product  could  be  associated  only  with  the 
design  and  production  costs.  On  the  other  hand,  if  a 
process  is  used  to  change  the  reliability  level,  the 
manufacturing  cost  includes  the  cost  to  the  manufacturer  of 
the  process. 

For  engines,  DOD  sets  the  terms  of  the  reliability 
requirement.  During  the  early  acquisition  phases, 
engineering  efforts  integrate  safety,  reliability, 
producibility ,  cost,  and  etc.  into  the  engine  design  (71). 


The  goal  is  to  achieve  an  acceptable  balance  among 
operational,  economic,  and  logistic  factors  (17:192).  Once 
the  baselined  design  is  established,  the  design,  materials, 
and  manufacturing  processes  of  the  engines  are  constantly 
changed  in  an  effort  to  increase  reliability  (37:6). 

To  increase  reliability,  one  method  is  the  Component 
Improvement  Program  (CIP).  A  function  of  the  CIP  is  to 
generate  Engineering  Change  Proposals  (ECP's)  (22:15).  The 
purpose  of  these  ECP's  may  be  to  correct  problems  (such  as 
wear,  failure,  degradation  of  engine  performance)  that  are 
detected  once  the  system  has  been  fielded  (63;  71). 

Another  method  of  increasing  reliability  has  been 
changing  the  manufacturing  process.  As  an  example, 
increases  in  technology  have  enhanced  the  bolting 
applications  for  engines.  At  one  time  bolt  holes  were  made 
in  two  engine  parts  and  then  the  parts  were  held  together 
with  bolts.  The  bolt  holes  were  placed  where  stress  on  the 
engine  which  could  led  to  failures.  A  new  process,  "inertia 
bonding"  is  now  used  to  weld  the  engine  parts  together. 

This  process  decreased  the  probability  for  the  type  of 
failures  caused  by  the  earlier  bolt  holes,  thus  it  increased 
reliability  (63). 

As  explained  in  chapter  two,  the  manufacturer  can 
trade-off  manufacturing  cost  and  warranty  claim  cost.  The 
manufacturing  cost,  the  cost  to  the  manufacturer  of  building 
a  given  reliability  into  the  product,  is  needed  in  order  to 
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measure  warranty  costs. 


To  determine  the  manufacturing 


costs  detailed  engineering  and  financial  information  with 
regard  to  the  production  process  is  required  (43:10). 

Obtaining  the  Cost  of  Building  Reliability. 

Prior  to  the  Warranty .  Prior  to  the 
warranty,  information  pertaining  to  the  contractor’s  costs 
of  production,  is  provided  by  the  contractor  and  estimated 
by  the  government.  This  information  is  used  to  establish 
the  basis  for  negotiation  of  contract  prices  (46:30;  69). 

A  contractor's  proposal,  describes  the  manner  in  which 
the  product  will  be  produced.  In  addition,  to  justify  the 
price,  the  contractor  intends  to  charge,  a  detailed  estimate 
of  proposed  or  anticipated  costs  are  described.  If  the 
contractor  has  not  produced  similar  products  before, 
extensive  technical  estimates  are  used  to  develop  the 
production  costs.  On  the  other  hand,  if  the  contractor  has 
experience  in  making  similar  products,  cost  information  is 
usually  estimated  using  the  contractor's  accounting  systems 
(46:84-88) . 

The  government  prepares  independent  cost  estimates  as  a 
measure  to  compare  the  contractor's  proposed  prices  (46:30). 
The  government  estimates  are  prepared  using  engineering 
analysis  of  the  item  to  be  procured  and  are  supported  with 
historical  data  on  the  cost  of  producing  similar  items 
(46:30) . 
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For  engines,  historical  cost  and  engineering  data  is 
available  (66).  The  data  is  used  by  specialist  in  the 
Program  Control,  Engineering,  and  Logistics  sections  of  the 
Deputy  for  Propulsion  organization,  and  other  offices  to 
develop  the  government  estimates  for  negotiations  (15;  66; 
69).  During  negotiations  the  estimates  are  used  by  contract 
negotiators  in  support  of  the  contracting  officer  for 
determining  what  the  contract  price  will  be  (15;  69). 
Although  the  data  is  used  for  technical  evaluations,  to 
determine  estimates  for  production  cost,  a  methodology  of 
using  the  data  to  evaluate  warranty  cost  does  not  presently 
exist  ( 66 ;  69 ) . 

One  reason  for  this,  according  to  a  contract  price 
analyst  who  has  experience  with  pricing  engine  warranties, 
is  that  the  process  to  determine  the  cost  of  building 
reliability  into  an  engine  would  involve  a  great  amount  of 
time  and  effort.  It  would  include  finding  and  collecting 
the  documents  relating  to  reliability  and  cost  (i.e., 
contracts,  technical  engineering  data,  cost  documents,  ECP 
and  CIP  data,  etc.),  then  determining  the  relationship  of 
costs  to  the  changes  in  reliability.  This  approach  of 
collecting  and  formatting  the  data  to  use  for  evaluating 
warranty  cost  has  not  been  considered  because  the  time  and 
effort  to  perform  such  an  analysis  may  not  be  worth  the 
benefits  (69).  Since  the  manufacture's  cost  of  building 
reliability  into  the  product  influences  the  manufacturer's 


total  cost,  knowing  this  information  would  enhance  the 
government's  ability  to  determine  the  warranty  price. 

After  the  Warranty  Expires .  After  the 
warranty  expires,  the  type  of  data  available  to  the 
government,  is  the  result  of  the  contract  requirements.  As 
an  example,  several  cost  reports  were  required  from  the 
contractor  for  the  F100  Lot  IX  engines.  One  report,  the 
Cost  of  Sales  Report,  was  generated  monthly  by  the 
contractor  and  provided  actual  manufacturing  costs  (22:1; 
76).  Other  reports  were  the  Cost  Information  Reports, 
generated  annually,  which  provided  quality  control, 
indirect,  and  tooling  costs  in  addition  to  the  manufacturing 
costs  ( 24) . 

Although  the  cost  reports  could  be  used  to  determine 
manufacturing  costs,  other  costs  that  impact  reliability 
(such  as  contractor's  costs  of  a  change  in  the  production 
process  and/or  an  ECP)  would  have  to  be  obtained  from  other 
sources.  In  order  to  determine  the  cost  to  the  contractor 
of  building  reliability  into  the  engines,  it  would  take  an 
understanding  of  both  the  contractor's  cost  accounting 
system  and  all  the  processes  that  changed  the  engines 
reliability  (57;  69;  71;  76). 

A  method  suggested  to  obtain  the  manufacturer's  cost  of 
building  reliability  into  the  product  was  to  require  the 
data  in  a  specified  format  to  be  provided  by  the  contractor 
as  a  condition  of  the  contract.  According  to  a  financial 
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specialist  in  the  Deputy  for  Propulsion  organization,  this 
method  would  require  additional  cost  to  the  government  and 
should  be  compared  with  the  benefits  of  the  data  (76). 

POD 1 s  Costs .  For  DOD,  there  are  two  costs  that  must  be 
obtained  in  order  to  measure  the  warranty  cost,  explicit 
cost  and  implicit  cost. 

Explicit  Cost .  As  described  in  chapter  two,  the 
explicit  cost  for  DOD  is  the  price  paid  for  the  warranty. 

It  is  equal  to  the  manufacturer's  total  cost  plus  the 
manufacturer's  profit. 

What  is  Required.  It  is  necessary  to  know 
the  warranty  price  in  order  to  evaluate  the  cost 
effectiveness  of  a  warranty.  What  DOD  pays  for  the  warranty 
will  depend  on  the  negotiating  power  of  DOD  with  the 
manufacturer  (43:9).  Since  the  manufacturer's  profit  is 
included  in  the  cost  of  the  warranty,  it  might  be  possible 
for  DOD  to  negotiate  an  increased  warranty  coverage  at  no 
additional  cost,  but  instead,  at  a  reduction  in  the 
manufacturer’s  profit. 

Obtaining  the  Explicit  Cost . 

Prior  to  the  Warranty .  Prior  the  the 
warranty,  there  are  at  least  three  methods  of  estimating  the 
explicit  cost  (23:3-1-3-10;  81:39-40).  One,  rule  of  thumb; 
two,  cost  estimating  relationships;  and  three,  the  bottom-up 
approach.  These  methods  provide  an  estimate  of  what  DOD 
should  pay  for  a  warranty,  however  there  are  problems  with 


the  methods . 


The  first  explicit  cost  estimating  method,  rule  of 
thumb,  is  the  development  of  a  ratio  of  one  cost  to  another. 
It  is  simple,  quick,  and  generalized.  However,  it  yields 
imprecise  estimates  and  it's  oversimplification  may  obscure 
relevant  details  that  make  its  use  inappropriate.  An 
example  of  the  rule  of  thumb  method  for  an  engine  warranty 
is  the  F100  Lot  IX  Supply  Warranty.  The  warranty  cost  was 
based  on  a  percentage  of  the  manufacturing  costs  (8). 

The  second  method  of  estimating  explicit  cost  is  the 
cost  estimating  relationships  (CER) .  The  CER  is  a 
statistically  developed  parametric  relationship  of  cost 
elements  that  drive  the  warranty  cost.  Although  the 
estimate  can  provide  a  firm  statistical  basis  for 
projections  and  predictions,  it  needs  extensive  data 
requirements  and  extensive  development  efforts.  Presently, 
since  there  have  only  been  a  limited  number  of  engine 
warranties,  there  is  lack  of  an  extensive  data  base  in  order 
to  use  this  method  for  engine  warranties  (66). 

The  third  explicit  cost  estimating  method,  the  bottom- 
up  approach,  identifies  all  warranty  costs  elements  required 
to  estimate  costs  and  summarizes  them  into  a  total  cost.  It 
requires  a  "build  up"  of  costs  that  may  include  elements  of 
costs  based  on  historical  data,  rule  of  thumb,  or  cost 
estimating  relationships.  The  bottom-up  approach  has 
distinct  advantages  in  that  it  can  provide  a  very  accurate 
cost  estimate.  However,  it  can  be  time  consuming,  complex, 


and  expensive  to  develop.  In  addition,  because  of  the 
detail,  the  data  needs  to  be  current,  accurate,  and 
complete.  An  example  of  the  bottom-up  method  for  an  engine 
warranty  is  the  F100  Lot  IX  Expanded  Engine  Warranty.  The 
warranty  cost  was  based  on  estimates  of  incremental  costs  to 
the  contractor  for  the  warranty  coverage  (8).  As  an 
example,  the  increase  in  the  contractor's  cost  of  material 
and  labor,  in  order  to  remedy  failures  projected  to  occur 
during  the  warranted  period,  was  estimated.  For  some 
warranties  this  approach  is  not  possible  to  use,  however  in 
this  case,  the  engines  had  been  in  production  for  eight 
years  and  there  was  historical  data  to  support  the  estimate. 

After  the  Warranty  Expires .  After  the 
warranty  expires,  the  price  paid  for  the  warranty  can 
sometimes  be  obtained  as  a  line  item  on  the  contract  or  from 
the  contracting  files  (58;  66).  The  price,  the  explicit 
cost  of  the  warranty,  should  be  the  manufacturer's  total 
cost  plus  the  manufacturer's  profit. 

For  the  F100  Lot  IX  engine  warranties,  the  price  paid 
was  documented  in  the  contract  files  (28).  Table  V  shows 
the  prices  paid  for  the  three  warranties.  The  Supply 
Warranty  covered  failures  in  material  and  workmanship;  it 
was  negotiated  as  a  percentage  of  manufacturing  cost  and  was 
priced  at  $6,073,229.  By  basing  the  price  on  a  direct  cost, 
the  contractor's  cost  and  the  contractor's  profit,  if  any, 
were  hidden  in  the  overall  procurement  costs. 


TABLE  V 


Explicit  Costs  for  F100  Lot  IX  Engine  Warranties 


Warranty 


Explicit  Cost 


Price  Contractor's  Contractor's 

Govt.  Paid  Costs  Profit 


Supply  $  6,073,229  ? 

High  Pressure  Turbine  $  -0-  7 

Expanded  Engine  $12,650,000  $4,914,000 


$7,736,000 


As  shown  in  Table  V,  the  High  Pressure  Turbine  Warranty 
which  covered  failures  of  the  high  pressure  turbines  was 
offered  to  the  government  for  no  change  in  the  contract 
price.  When  the  contractor  offered  this  warranty,  it  was  a 
change  to  the  existing  Lot  IX  procurement  contract.  A 
government  evaluation  of  the  warranty  was  completed.  The 
expected  number  of  failures  and  the  costs  to  repair  the 
failures  were  considered  in  order  to  determine  the  value  of 
the  warranty.  The  value  was  estimated  to  fall  between  2.5 
and  7.4  million  dollars  (56:  Tab  D).  However,  the 
contractor  was  not  required  to  provide  warranty  cost  data, 
so  the  actual  cost  and  profit  to  the  contractor  could  not  be 
obtained . 

The  Expanded  Engine  Warranty  covered  failures  of  the 
engines,  modules,  and  accessories;  it  was  negotiated  for  a 
price  of  $12,650,000,  as  shown  in  Table  V.  With  this 
warranty,  the  contractor  was  required  to  provide  cost  data. 


The  data,  a  report  titled  Contractor  Warranty  Cost  Report, 
showed  that  the  contractor's  warranty  cost  was  $4,914,000. 
Subtracting  the  warranty  cost  from  the  price  paid  for  the 
warranty,  leaves  the  contractor's  warranty  profit.  In  this 
case,  the  profit  was  $7,736,000. 

Implicit  Costs .  As  described  in  chapter  two,  the 
implicit  DOD  cost  of  warranties  are  the  internal  costs  of 
enforcing  the  warranty,  the  cost  of  documenting  and 
complying  with  the  claim  provisions. 

What  is  Required .  Identification  and 
measuring  of  the  implicit  costs  are  necessary  in  order  to 
evaluate  the  cost-effectiveness  of  warranties.  The  Warranty 
Handbook  divides  implicit  costs  into  direct  costs  and 
indirect  costs,  depending  on  the  relationship  of  the  cost 
with  the  warranty.  Direct  costs  includes  those  costs  for 
warranty  development  and  administration,  obtaining  and 
providing  special  data,  warranty  training,  warranty 
monitoring,  and  special  transportation  (13:3-14). 

Indirect  costs  include  those  costs  that  increase  spares 
requirements  because  of  longer  pipelines,  and  also,  reduce 
self-sufficiency  by  having  the  contractor  repair  failures  as 
opposed  to  enhancing  organic  repair  capabilities  (13:3-14). 

The  implicit  costs  are  also  known  as  the  government 
administrative  costs  (81:26,27).  For  engine  warranties,  the 
implicit  cost  can  include,  but  not  be  limited  to,  data 
management  costs,  contractor/Air  Force  interface  costs. 


warranty  administration  training  costs,  and  transportation 
costs  (23:111-29;  57;  72;  81:27).  These  implicit  costs  are 
discussed  below. 

Data  Management  Costs .  Data  management 
costs  includes  the  labor,  material,  and  computer  costs  to 
document  and  support  warranty  claims.  These  costs  may  be 
incurred  from  the  following  activities: 

(1)  development  and  implementation  of  a  manual  and/or 
computer  system  to  track  warranted  items; 

(2)  collecting  information  about  a  failure  incident  to 
determine  and  document  the  failures  covered  by  the  warranty; 
and 

(3)  preparing  and  processing  forms  in  order  to  file 
the  warranty  claim. 

Contractor /Air  Force  Interface  Costs . 

The  contractor/Air  Force  interface  costs  includes  the  labor 
and  material  costs  for  government  personnel  to  monitor  and 
manage  the  warranty.  The  functions  under  this  cost  may 


include  the  following: 

(1)  liaison  between  the  program,  support,  user,  and 
contractor  activities; 

(2)  determining  warranty  compensation;  and 

(3)  preparing  and  initiating  an  implementation  plan. 

Warranty  Administration  Training  Costs . 
The  warranty  administration  training  costs  includes  the  cost 
of  training  personnel  to  operate,  maintain,  and  support  the 


warranted  items  so  as  not  to  void  the  warranty.  In 
addition,  the  cost  of  training  personnel  to  initiate  and 
process  warranty  claims  may  be  included. 


Transportation  Costs .  Transportation 
costs  include  the  cost  of  packaging,  handling,  and  shipping 
warranted  items  to  and  from  the  contractor. 

From  this  brief  discussion,  it  can  be  seen  that  there 
are  numerous  implicit  costs  that  can  be  considered  when 
evaluating  warranties.  In  order  to  do  the  evaluation 
accurately,  it  is  necessary  to  identify  and  collect  those 
costs  that  could  have  an  impact  on  the  systems  life  cycle 
cost . 

Obtaining  the  Implicit  Costs . 

Prior  to  the  ’Warranty .  Prior  to  the 
warranty,  the  implicit  cost  must  be  estimated.  It  is 
necessary  to  identify  and  determine  the  costs  that  could 
have  an  impact  on  the  systems  life  cycle  cost  (13:7-7). 

Presently  there  is  disagreement  about  the  significance 
of  implicit  costs  to  the  warranty  cost  effectiveness 
decision.  As  an  example,  on  one  hand,  a  task  force  team, 
consisting  of  functional  specialist  from  within  the 
Aeronautical  Systems  Division  (ASD)  and  the  Air  Force 
Acquisition  Logistics  Center,  believed  that  government 
administrative  cost  may,  generally,  be  insignificant  (9:27) 
Although  there  was  no  data  to  support  their  claim,  the 
believe  was  that  the  implicit  cost  would  seldom  drive  the 


decision  in  the  warranty  cost  effectiveness  determination 
(9:27). 


On  the  other  hand,  in  the  report.  Cost  Effectiveness 
Study  USAF  Jet  Engine  Warranty  Programs :  The  Alternate 
Fighter  Engine  Warranty,  administrative  cost  were  not 
considered  negligible  (23:111-29).  The  report  discussed 
several  implicit  cost  considerations  for  the  Alternate 
Fighter  Engine  Warranty  (the  F100-PW-220  and  F110-GE-100 
engines).  The  implicit  costs  included  the  transportation 
costs  of  shipping  engines  from  operational  bases  to  either 
contractor  or  government  repair  facilities,  and  also,  the 
costs  of  filing  the  warranty  claims.  In  addition,  methods 
to  estimate  the  costs  were  described.  The  conclusion  was 
that  the  implicit  cost  could  impact  the  life  cycle  cost  of 
the  engines  (23:111-25-111-33). 

Actual  implicit  costs  data  has  not  been  collected  and 
analyzed  to  validate  either  viewpoint.  An  effort  was  being 
conducted  by  an  ASD  organization  to  develop  an  approach  for 
estimating  the  Government's  costs  (9:26?  72);  but  the  effort 
has  subsequently  been  canceled  due  to  other  organizational 
priorities  (72).  The  Cost  Analysis  Office,  Directorate  of 
Aeronautical  Systems  Division  (ASD)  recommends,  for  ASD 
products  being  procured,  an  estimate  for  government 
administrative  cost  to  be  used  when  conducting  a  cost- 
benefit  analysis  (72).  The  estimate  for  contracts 
containing  warranties,  is  a  cost  of  $55,000  per  operational 


base  for  each  of  the  three  years  following  deployment  of  the 
system.  The  estimate  is  based  on  experience  with  warranties 
on  aircraft  engines  and  the  following  criteria  (9:26;  72): 

(a)  Contract  quantity  orders  of  100-200  units. 

(b)  Supply/Maintenance  guarantee  followed  by  shop 
visit  rate  guarantee  (most  frequently  used  on 
engine  contracts). 

(c)  Product  deliveries  1-2  years  after  contract  award. 

(d)  Payback  beginning  after  fielding  with  major 
payback  first  3  years  after  fielding. 

(e)  Warranty  cost  is  a  function  of  the  number  of 
engines  on  contract. 

(f)  Shop  visit  rate  guarantee  payback  is  based  on  the 
number  of  actual  shop  visits  per  1000  engine 
operating  hours  calculated  at  12  month  intervals. 

After  the  Warranty  Expires.  After  the 
warranty  expires,  the  implicit  costs  may  not  be  easy  to 
obtain  because  the  costs  are  not  tracked  and  collected  (72). 


For  the  F100  Lot  IX  engine  warranties,  the  implicit 
costs  have  already  been  incurred.  However,  the  costs  have 
not  been  measured  in  order  to  evaluate  the  warranty.  Three 
of  the  implicit  costs  for  the  F100  Lot  IX  warranties  will  be 
described.  One  implicit  cost  was  the  transportation  cost  of 
shipping  warranted  items  between  the  operational  base  and 
the  contractor's  repair  facilities.  A  second  implicit  cost 
was  the  cost  of  changes  made  to  the  USAF  Material  Deficiency 
Reporting  and  Investigating  System  to  include  the  warranty 
claim  procedures.  A  third  implicit  cost  was  the  additional 
costs  incurred  by  the  Air  Force  when  components  and  modules 
were  misrouted  to  the  depot  where  government  repair  voided 
the  warranty  coverage.  These  implicit  costs  will  be 
examined  separately. 
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Transportation .  The  contract  for 
the  F100  Lot  IX  warranties  required  the  government  to  pay 
the  transportation  costs  of  any  warranted  item  returned  to 
or  sent  from  the  contractor's  plant.  These  transportation 
costs  would  be  difficult  to  measure  because  there  is  no 
existing  data  system  that  collects  these  costs  (25:30;  66). 

A  method  to  obtain  the  cost  would  be  to  identify  the  items 
shipped,  determine  the  route  and  mode  of  shipping,  and 
calculate  the  transportation  cost.  This  method  is  similar 
to  the  approach  used  in  the  report,  "Cost  Effectiveness 
Study  USAF  Jet  Engine  Warranty  Programs:  The  Alternate 
Fighter  Engine  Warranty, "  to  estimate  the  shipping  cost  of  a 
F100-PW-220  engine  (23:  III-25-III-27 )  . 

Warranty  Claim  Procedures .  The 
claim  procedures  used  for  the  F100  Lot  IX  warranties,  in 
order  to  report  a  warranted  failure,  were  implemented  as  a 
change  to  the  established  Technical  Order  (TO)  00-35D-54, 
USAF  Material  Deficiency  Reporting  and  Investigating  System 
(38).  A  brief  discussion  of  the  TO  00-35D-54  reporting 
procedure  and  the  process  used  to  claim  a  warranty  failure 
follows . 

Historically,  prior  to  the  utilization  of  warranties  on 
engine  contracts,  the  reporting  procedures  under  TO  00-35D- 
54  were  activated  to  report  deficiencies  such  as  design, 
maintenance,  material,  quality,  and  software.  The  purpose 
of  reporting  these  deficiencies  under  TO  00-35D-54  has  been 
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to  provide  data  from  the  using  activity  to  the  engine 
program  manager.  The  data  is  used  to  determine  the  reason 
for  the  deficiency,  discover  if  trends  are  occurring,  and 
also,  to  initiate  fixes  for  the  deficiency  (30:1-1;  79). 

According  to  the  TO  00-35D-54,  deficiencies  must  be 
reported  using  either  a  Service  Report  (when  the  engine 
program  management  is  with  the  Deputy  for  Propulsion 
organization)  or  a  Material  Deficiency  Report  (when  the 
engine  program  management  responsibility  has  been 
transferred  to  an  Air  Logistic  Center).  With  the 
utilization  of  warranties  on  engine  contracts,  the  TO  00- 
35D-54  reporting  procedures  were  supplemented  in  order  to  be 
used  to  initiate  warranty  claims  (38). 

The  process  by  which  F1G0  Lot  IX  warranted  failures 
were  claimed  is  illustrated  in  Figure  8.  The  warranty  claim 
process  began  with  the  using  activity  (A  in  Figure  8) 
discovering  a  deficient  engine,  module,  or  component.  Next 
the  using  activity  documented  the  deficiency  on  a  Service 
Report  (SR)  or  a  Material  Deficiency  Report  (MDR)  and  held 
the  deficient  item,  called  an  exhibit,  for  disposition 
instructions.  As  shown  in  Figure  8,  the  SR  or  MDR  was  sent 
to  the  contact  point  (B  in  Figure  8) .  The  contact  point  was 
the  office  which  controlled  and  routed  the  SR's  or  MDR's  to 
the  warranty  administrator. 

The  warranty  administrator  (C  in  Figure  8),  with  the 
engine  manufacturer  representative  (D  in  Figure  8), 
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claim  valid, 


determined  the  warranty  claim  decision  (i.e., 
who/where  to  make  corrections  in  accordance  with  the 
contract,  etc.).  Then  the  administrator  distributed  exhibit 
disposition  instructions  to  the  using  activity.  The  exhibit 
was  shipped  to  the  repair/replace  activity  (E  in  Figure  8) 
where  appropriate  action  was  taken.  A  repaired  or  replaced 
item  was  returned  to  the  using  activity  (from  E  to  A  in 
Figure  8).  A  findings  (or  investigation)  report  was  made  of 
the  action  taken  to  complete  the  warranty  claim 
documentation.  In  addition,  the  using  activity  was  sent  a 
message  of  the  action  taken. 

For  the  F100  Lot  IX  warranties,  the  warranty  claim 
procedures  costs  included  the  cost  of  supplementing  the  TO 
00-35D-54  reporting  procedures.  One  of  the  implicit  costs 
was  training  cost  to  make  sure  the  procedures  were 
understood  and  properly  implemented.  Another  implicit  cost 
may  have  been  cost  for  warranty  administrators.  With  the 
additional  type  of  warranties  (the  High  Pressure  Turbine  and 
the  Expanded  Engine  Warranties  were  first  introduced  with 
the  F100  Lot  IX  engines)  there  was  an  increase  in  the  type 
of  claims,  which  meant  additional  workload  in  administering 
the  warranties.  These  costs  and  others  associated  with  the 
warranty  claim  procedures  need  to  be  identified  and  priced 
in  order  to  measure  the  implicit  cost  for  the  F100  Lot  IX 


warranties . 


Voided  Warranties .  The  contract  for 
the  P100  Lot  IX  engine  warranties,  required  that  all  repairs 
were  to  be  made  at  the  contractor's  facilities  rather  than 
the  government  depot.  During  the  course  of  the  warranty 
period  some  modules  and  components  were  sent  to  the  depot 
where  government  repair  voided  the  warranty  coverage  (55; 
75).  The  voided  warranties  increased  the  implicit  costs  in 
that  the  government  incurred  unnecessary  repair  costs.  A 
method  to  obtain  the  implicit  cost  of  voided  warranties 
would  be  to  identify  the  items  repaired  and  determine  what 
costs  were  incurred.  This  method  is  similar  to  the  approach 
used  in  an  Air  Force  Audit  Report  to  estimate  the  the  voided 
warranty  costs  incurred  by  the  government  as  of  October  1984 
for  the  F100  High  Pressure  Turbine  Warranties  (2:6).  In  the 
audit  report  it  was  stated,  the  warranty  on  81  high  pressure 
engine  turbines  had  been  voided  and  the  government  had 
incurred  costs  upto  $2.5  million  to  inspect,  service,  or 
repair  the  items. 

Another  implicit  cost,  incurred  for  the  F100  Lot  IX 
engine  warranties,  was  the  additional  pipeline  in  shipping 
and  repair  times  of  items  sent  to  be  repaired  at  the 
contractor's  facilities  instead  of  the  government  depot 
(81:17) . 

Although  the  implicit  costs  for  the  F100  Lot  IX 
warranties  have  already  been  incurred,  these  costs  would  not 
be  easy  to  obtain.  Without  complete  knowledge  of  which 


implicit  costs  incurred  and  collection  of  those  costs,  it  is 
not  possible  to  measure  the  implicit  cost  for  the  F100  Lot 
IX  warranties. 


Evaluating  Warranty  Benefits 

In  chapter  two  the  information  required  to  evaluate  the 
benefits  of  DOD  warranties  was  identified.  The  benefit  of 
warranties  was  defined  as  the  value  of  the  increased 
reliability  as  a  result  of  a  warranty.  As  explained  in 
chapter  two,  in  order  to  evaluate  warranty  benefits,  one 
must  know  the  change  in  reliability,  before  and  after  the 
warranty,  and  the  value  of  the  change  in  an  operational 
context  to  DOD. 

Measuring  Warranty  Benefits 

To  measure  the  warranty  benefit,  DOD  must  obtain  or 
estimate  the  benefits  that  were/will  be  a  result  of  the 
warranty . 

What  is  Required.  The  DOD  benefits  associated  with  a 
warranty  must  be  identified  and  defined.  Benefits  may  be 
quantitative  or  qualitative.  The  quantitative  benefits  are 
normally  stated  in  terms  of  controlling  or  improving 
equipment  quality,  availability,  reliability, 
supportability ,  support  or  repair  costs,  and  etc.  The 
qualitative  benefits  are  normally  stated  in  terms  of  the 
quality  of  the  products  delivered,  and  the  contractor's 
incentive  provided  by  the  warranty  in  order  to  increase  the 
product's  reliability  (13:7-12;  31:7,30). 


Quantitative  Benefits .  Quantitative  benefits  can 


be  measurements  of  improved  performance,  reliability,  and 
quality.  The  improvements  can  be  quantified  through  the  use 
of  reliability  and  maintainability  parameters  such  as  Mean 
Time  Between  Failure  (MTBF)  and  Mean  Time  To  Repair  (MTTR) . 
In  addition,  quantitative  benefits  can  be  measured  as 
reductions  in  life  cycle  costs.  For  example,  increased 
reliability  means  less  failures,  which  influences  logistics 
and  support  elements  associated  with  the  failures.  These 
elements  include  spares  levels,  maintenance  manpower  levels, 
and  material  costs  for  repair  (13:7-12;  31:7). 

Qualitative  Benefits .  Qualitative  benefits  are 
not  measurable  in  quantitative  terms,  but  instead  are  based 
on  subjective  assessments.  Qualitative  warranty  benefits 
may  include  (13:7-12;  31:7): 

(a)  motivating  the  contractor  to  design  the  system  to 
meet  requirements  at  initial  production  release  and  to 
operate  as  intended  in  the  field; 

(b)  early  and  rapid  resolution  of  problem  areas  due  to 
high  visibility  and  management's  attention; 

(c)  incentive  for  the  contractor  to  introduce  no-cost 
engineering  change  proposals  in  order  to  reduce  the  number 
of  repairs  during  the  warranty  period;  and 

(d)  more  realistic  estimates  of  field  performance 
which  permit  improved  accuracy  in  planning  operation  and 


support  resources. 


Obtaining  the  Warranty  Benefits . 


Prior  to  the  Warranty .  Prior  to  the  warranty  the 
benefits  are  estimated.  The  estimates  can  be  based  on 
engineering  calculations  of  the  number  of  failures  that  are 
predicted  to  occur  during  the  warranty  period  (9:31). 

For  engines,  the  projected  number  of  failures  are 
determined  by  the  technical  evaluation  of  the  engineering 
function.  The  benefits  are  the  value  of  the  material, 
labor,  and  services  associated  with  taking  corrective  action 
for  the  projected  failures  (66). 

After  the  Warranty  Expires .  After  the  warranty 
expires,  the  change  in  reliability  before  and  after  the 
warranty  must  be  measured  in  order  to  evaluate  the 
effectiveness  of  the  warranty. 

For  the  weapon  systems  that  use  the  F100  engines,  an 
essential  performance  reliability  parameter  is  the 
unscheduled  engine  removal  (UER) .  This  parameter  is  a 
measurement  of  any  inherent  failure  that  resulted  in  an 
engine  removal  and  is  affected  by  the  warranted  parameter  of 
engine  failures  within  200  hours  of  total  operating  time. 
This  type  of  warranted  parameter  was  used  on  the  F100  Lot  IX 
engines . 

Shown  in  Table  VI  is  the  UER  parameter  of  the  F100 
engines  for  Lot  XIII  (before  the  warranty)  and  Lot  IX  (after 
the  warranty) .  It  can  be  seen  that  the  UER  reliability 
increased  for  both  the  F100-PW-100  and  F100-PW-200  engines 
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TABLE  VI 


UER/1000  Engine  Flying  Hours  Reliability 
at  200  Hours  Total  Operating  Time 


F-15  (used  two 
F100-PW-100 
engines ) 


F-16  (used  one 
F100-PW-200 
engine ) 


Lot  VIII 

(before  the  2.1  4.4 

warranty) 


Lot  IX 

(after  the  1.7  3.8 

warranty) 


from  Lot  XIII  to  Lot  IX.  However,  an  engineer  from  the 
Deputy  of  Propulsion  organization  stated  that  the  UER 
reliability  increases  are  more  likely  due  to  the  Component 
Improvement  Program  and  normal  maturity  of  the  engine 
systems  than  to  the  warranty  (71). 

Some  of  the  Component  Improvement  Program  (CIP) 
initiatives  implemented  during  the  period  of  the  F100  Lot  IX 
warranty  were: 

(a)  improved  wear  resistance  of  boost  piston: 

(b)  incorporation  of  vane  stage  and  boost  stage 
improvements ; 

(c)  increased  1st  and  2nd  turbine  blade  durability 

(22)  . 

These  and  other  reliability  improvements  could  have 
changed  the  UER  rate  (71).  Since  reliability  is  not 


monitored  by  the  impacting  factors,  the  benefits 
attributable  to  the  CIP's  and  the  benefits  attributable  to 
the  warranty  are  unclear.  Without  knowing  and  measuring  the 
change  in  reliability  resulting  from  the  warranty,  the 
benefits  can  not  used  to  evaluate  the  warranty. 

In  this  chapter  results  of  the  research  have  been 
reported.  In  the  next  chapter  a  summary,  conclusions,  and 
recommendations  will  be  given. 


V. 


Summary ,  Conclusions ,  and  Recommendations 


The  purpose  of  this  chapter  is  to  present  the  summary, 
conclusions,  and  recommendations  of  this  thesis  effort.  The 
summary  will  be  an  overview  of  what  is  needed  in  order  to 
evaluate  the  costs  and  benefits  of  DOD  warranties.  Next  the 
conclusions  of  chapter  four  will  be  discussed. 

Specifically,  the  discussion  will  center  on  some  of  the 
problems  in  obtaining  the  information  required  for  assessing 
the  benefits  and  costs  of  jet  engine  warranties  using  the 
methodology  of  chapter  two.  Finally  the  recommendations  for 
future  research  will  be  outlined. 


Summary 


With  the  passage  of  Public  Laws  98-212  and  98-525 


requiring  cost  effective  warranties,  and  also,  the 
requirements  of  DOD  guidance,  the  costs  of  the  warranty  must 
be  compared  to  the  benefits  of  the  warranty  to  perform  the 


cost-benefit  analysis.  A  comparison  of  the  alternatives 
(i.e.,  with  a  warranty  and  without  a  warranty)  can  be  made 
to  determine  the  warranty's  cost  effectiveness.  A  structure 


for  evaluating  the  costs  and  benefits  of  a  warranty  is 
illustrated  in  Figure  9. 

As  shown  in  Figure  9,  the  costs  to  DOD  of  the  warranty 
is  the  price  of  the  warranty,  plus  the  cost  of  enforcing  the 
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POD  1 s  Costs 
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vs 
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Minus 

Limitations  of  Product  Use  to  Preserve  the  Warranty 


Evaluating  DOD ' s  Warranties: 
a  Cost  and  Benefit  Structure 


Figure  9 


warranty,  minus  the  reduction  in  operating  and  support 
costs.  The  price  of  the  warranty  is  a  function  of  the 
manufacturer's  total  cost  (the  manufacturer's  warranty  claim 
cost  and  the  manufacturing  cost)  and  DOD ' s  ability  to 
negotiate  a  fair  profit  with  the  contractor.  The  cost  of 
enforcing  the  warranty  is  associated  with  DOD ' s  implicit 
costs  including  the  costs  of  administration,  training,  and 
transportation.  The  reduction  in  operating  and  support  cost 
is  due  to  fewer  failures  and  the  repairs  cost  that  would 
have  been  associated  with  the  failures  had  a  warranty  not 
been  utilized.  With  a  warranty,  the  contractor  partially 
absorbs  these  repair  costs. 

The  benefits  to  DOD,  as  shown  in  Figure  9,  is  the 
increase  in  product  availability  due  to  a  higher  product 
reliability,  minus  any  reduction  in  availability  due  to 
sending  the  product  bach  to  the  contractor  for 
repair/replacement,  minus  any  limitations  on  use  of  the 
product  in  order  to  preserve  the  warranty. 

It  is  important  to  establish  a  structure  for  separating 
the  costs  and  benefits  in  order  to  ensure  that  all  costs  and 
benefits  are  accounted  for.  In  addition,  it  is  suggested 
that  all  the  costs  for  the  warranty,  whether  cost  additions 
or  reductions,  be  compared  to  all  benefits,  whether  benefit 
additions  or  reductions,  to  ensure  that  all  costs  and 
benefits  are  included  in  the  analysis  and  are  not  double 
counted. 
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From  this  summary  of  evaluating  the  costs  and  benefits 
of  a  warranty,  a  framework  can  be  establisned  for  discussing 
the  information  that  is  required  to  be  obtained.  The 
framework  requires  that  the  manufacturer's  and  DOD ' s  costs, 
and  also,  DOD ' s  benefits  be  obtained.  Some  of  the  problems 
found  in  obtaining  this  information  for  assessing  jet  engine 
warranties  were  analyzed  in  chapter  four.  The  following 
conclusions  are  based  on  the  analysis  accomplished  in 
chapter  four. 

Conclusions 

The  conclusions  of  chapter  four  can  be  separated  into 
three  sections.  The  first  section  is  that  of  the 
manufacturer's  total  costs.  This  information,  known  or 
estimated,  enhances  DOD ' s  negotiating  power  for  the  warranty 
price.  The  second  section  is  DOD ' s  costs  which  includes 
explicit  cost  (influenced  by  the  manufacturer's  total  costs) 
and  implicit  costs.  The  third  section  is  DOD's  benefits 
which  are  attributable  to  the  warranty.  Table  VII  is  a 
summary  of  some  problems  found  during  this  research  effort. 
Each  problem  will  be  discussed  below. 

Manufacturer ' s  Total  Costs .  For  DOD  it  is  important  to 
know  the  manufacturer's  total  cost  of  the  warranty  in  order 
to  determine  the  price  of  the  warranty,  DOD's  explicit  cost. 
The  analysis  in  chapter  four  revealed  that  these  costs  are 
generally  not  available  in  the  Air  Force  records.  The 


Table  VII 


Conclusions  Summarized: 
Problems  in  Measuring  Costs  and 
Benefits  of  Jet  Engine  Warranties 


Manufacturer  1 s  Total  Cost 
Warranty  Claim  Cost 

Manufacturer's  Cost  of  a  Failure 

Difficult  to  measure  unless  data  provided  by 
contractor 

Not  all  costs  are  traceable  in  AF  records 

Number  of  Failures 

Difficult  to  estimate  if  contracted  parameter  does 
not  equate  to  maintenance  data 
Failure  data  from  contractor  does  not  include  all 
failures 

Manufacturing  Costs 

Engine  manufacturing  cost  available,  however  not  used 
to  evaluate  warranty  costs 


POD ' s  Costs 
Explicit  Cost 

Based  on  negotiations,  sometimes  documented  price  is 
not  the  same  as  the  estimated  manufacturer's 
total  cost  plus  profit 

Implicit  Costs 

Disagreement  about  significance 

Must  know  what  the  implicit  cost  are  in  order  to 
measure 


POD ' s  Benefits 

Change  in  Reliability 

Difficult  to  determine  the  change  in  reliability 
attributable  to  the  warranty 


manufacturer's  total  cost  is  a  function  of  the  warranty 
claim  costs  and  the  manufacturing  costs. 


Warranty  Claim  Cost .  The  warranty  claim  cost  is 
made  up  of  the  manufacturer's  cost  of  a  failure  and  the 
number  of  failures. 

For  the  manufacturer's  cost  of  a  failure,  Table  VII 
shows  two  problems  that  were  exposed  in  chapter  four.  The 
first  is  the  difficulty  in  measuring  the  manufacturer's  cost 
of  a  failure  unless  the  data  is  provided  by  the  contractor. 
This  is  accomplished,  prior  to  the  warranty,  in  the 
contractor’s  proposal;  and  after  the  warranty  expires,  in 
contractor  provided  reports,  if  the  data  was  required  in  the 

1  terms  of  the  contract.  For  the  Air  Force,  these  cost  are 

important  from  the  standpoint  of  providing  information  that 
can  be  used  (a)  for  comparing  contractor's  cost  with 
previous  lots  (or  similar  systems),  (b)  to  support  the  price 
of  the  warranty,  and  also  (c)  to  evaluate  the  warranty  after 
it  expires. 

In  Table  VII,  the  second  problem  is  that  not  all  the 
contractor's  cost  are  traceable  in  Air  Force  records.  For 
engines.  Air  Force  maintenance  records  are  useful  to 
estimate  the  cost  to  correct  a  failure.  However,  at  least 
two  problems  occur  when  extensively  using  Air  Force  records 

i 

\  to  evaluate  the  contractor's  cost  of  a  warranty.  One 

problem  is  that  the  maintenance  data  may  not  be  accurate. 
Secondly,  the  estimate  will  only  include  the  cost  to 
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repair/replace  a  failure  and  not  consider  the  other 
contractor's  claim  costs  (i.e.,  preparing  reports, 
administration) . 

For  the  number  of  failures  two  problems  were 
encountered  in  chapter  four,  as  shown  in  Table  VII.  The 
first  problem  can  occur  prior  to  a  warranty,  when  the  data 
must  be  estimated.  Air  Force  maintenance  records  provide 
historical  failure  data  on  engines.  However,  if  the 
performance  measurement,  or  parameter,  being  warranted  is 
not  captured  in  the  maintenance  records,  estimating  the 
expected  number  of  failures  is  limited  to  operational 
testing  and  engineer's  technical  evaluations.  The 
difficulty  in  estimating  the  expected  number  of  failures 
contributes  to  the  risk  of  the  warranty  which  tends  to 
increase  the  warranty  price. 

The  second  problem  can  occur  after  the  warranty 
expires.  If  the  contractor  provided  a  report  of  failures 
claimed,  the  report  may  not  include  all  the  failures.  This 
is  especially  true,  if  some  of  the  failures  were  not 
claimed.  Unclaimed  failures  can  result  in  two  conditions. 
One  condition  is  that  the  warranty  does  not  motivate  the 
contractor  and  assure  a  reliable  product,  instead  the 
warranty  increases  the  contractor's  profit.  The  second 
condition  is  that  DOD  incurs  additional  internal  costs  to 
correct  the  failure,  costs  that  should  be  borne  by  the 
contractor . 
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Manufacturing  Cost .  Manufacturing  cost  is  the 


costs  of  building  reliability  into  the  product.  Knowing 
these  costs,  would  enhance  DOD's  ability  to  determine  the 
warranty  price.  The  problem  in  this  area,  discussed  in 
chapter  four  and  shown  in  Table  VII,  is  that,  although  the 
cost  of  building  a  product  and  the  reliability  level  can  be 
determined,  these  costs  are  not  being  used  to  evaluate 
engine  warranty  costs.  The  primary  reason  for  this  is  that 
the  data  is  not  collected  and  formatted  so  that  it  can  be 
utilized  for  warranty  evaluations.  A  contract  price  analyst 
who  has  experience  with  pricing  engine  warranties,  stated 
that,  the  time  and  effort  necessary  to  gather  and  put  the 
data  into  a  useful  format  may  outweigh  any  benefits  (69) . 

POP ' s  Costs .  DOD’s  costs  include  both  explicit  cost 
and  implicit  cost. 

Explicit  Cost.  Explicit  cost  is  the  price  paid 
for  a  warranty  and  is  a  function  of  the  manufacturer's  total 
cost  and  DOD's  negotiating  power.  The  problem  in  this  area, 
shown  in  Table  VII,  is  that  the  negotiated  price  (what  the 
government  paid  for  the  warranty)  is  not  always  based  on  the 
manufacturer's  cost  plus  its  profit.  Sometimes  it  is  based 
on  a  percentage  of  a  direct  cost.  In  this  case,  DOD  can  not 
determine  what  the  contractor's  cost  of  the  warranty  is  or 
the  level  of  profit,  if  any,  the  contractor  is  receiving. 
Instead  these  costs  are  hidden  in  the  overall  procurement 
costs  and  it  is  difficult  for  DOD  to  determine  if  the 
warranty  price  is  fair  to  both  the  contractor  and  DOO. 
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Implicit  Costs .  Implicit  costs  are  DOD's  internal 
costs  of  enforcing  the  warranty.  In  Table  VII  there  are  two 
problems  in  this  area  that  were  encountered  in  chapter  four. 
The  first  problem  was  the  disagreement  about  the 
significance  of  DOD's  internal  costs  of  a  warranty  to  the 
cost-benefit  analysis.  Although,  a  standard  rate  is 
suggested  for  the  administrative  (implicit)  cost  of 
Aeronautical  Systems  Division  products,  actual  implicit 
costs  data  has  not  been  collected  and  analyzed  to  develop  a 
comprehensive  estimate.  The  second  problem,  which  actuates 
the  arguments  regarding  the  significance  of  the  implicit 
cost,  is  that,  without  knowledge  of  what  implicit  costs  are 
incurred  due  to  a  warranty,  the  costs  can  not  be  collected 
and  measured. 

POD' s  Benefits .  DOD's  benefits  of  a  warranty  are  the 
value  of  the  increased  reliability  as  a  result  of  a 
warranty.  An  increase  in  reliability  means  fewer  failures 
and  fewer  failures  means  an  increase  in  availability. 
Therefore,  an  increase  in  reliability  has  an  impact  on  a 
systems  availability.  The  problem,  analyzed  in  chapter  four 
and  shown  in  Table  VII,  of  measuring  the  change  in 
reliability,  is  the  difficulty  in  separating  the  benefits 
attributable  to  a  warranty  and  the  benefits  attributable  to 
other  factors.  Unless  the  benefits  of  warranties  are 
identified,  defined,  and  measured  there  will  be 
complications  in  performing  cost-benefit  analyses  and 
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Recommendations  for  Future  Research 

During  the  course  of  this  thesis  effort  some  areas  for 
future  research  became  evident.  Included  were  the  five 
topics  which  are  discussed  below. 

First,  since  warranties  on  major  weapon  systems  are  for 
the  most  part  mandatory,  creating  a  data  base  of  the 
manufacturer's  costs  would  be  beneficial  for  future  warranty 
evaluations.  In  particular,  the  warranty  claim  cost  and  the 
cost  of  building  reliability  into  the  product  could  help  DOD 
in  future  warranty  negotiations.  Research  in  this  area 
could  provide  a  structure  of  what  data  to  collect  and  how  to 
format  it  so  that  the  data  could  be  used  for  assessing 
warranty  cost. 

Second,  an  analysis  of  the  relevant  implicit  costs 
should  be  conducted.  The  study  should  determine  what  cost 
categories,  if  any,  contribute  to  DOD's  cost  of  enforcing 
warranties.  In  addition,  the  analysis  could  validate  that 
the  government's  administrative  cost  are  either  significant 
or  they  are  not  significant  to  the  warranty  cost 
effectiveness  determination. 

Third,  a  study  of  the  relationship  between  warranties 
and  other  methods  used  to  improve  reliability  (i.e., 
Component  Improvement  Program,  or  Engineering  Change 
Proposals)  should  be  made.  The  purpose  would  be  to 
determine  the  benefits  (change  in  reliability)  attributable 


to  each  of  the  various  methods.  In  addition,  a  study  of  the 
impact  of  each  method  on  one  another  would  be  of  value. 

Fourth,  now  that  warranties  must  be  included  in 
procurement  contracts  for  weapon  systems,  there  should  be  an 
examination  of  the  impact  of  multiple  warranties  on  a  weapon 
system.  Research  in  the  this  area  could  address  how  a 
warranty  on  an  major  subsystem  (i.e.,  aircraft  engines)  and 
the  warranty  on  the  overall  system  (i.e.,  the  aircraft 
itself)  impact  each  other. 

Fifth,  because  of  the  problems  associated  with 
warranting  reliability  parameters  and  keeping  maintenance 
data  as  historical  records,  an  examination  of  translating 
the  maintenance  data  into  reliability  parameters  would  be 
useful.  A  possible  starting  point  is  with  AFA L.C/ERRR,  the 
Reliability  and  Maintainability  Directorate  of  the  Air  Force 
Acquisition  Logistics  Center,  who  suggested  that  such  an 
examination  is  needed  (20). 

Closing  Comments 

A  cost-benefit  analysis  is  conducted  to  determine 
whether  use  of  a  warranty  would  be  cost  effective.  In 
addition,  a  cost-benefit  analysis  is  required  to  be 
performed,  and  should  be  documented  in  the  contract  files. 
Today,  there  is  not  a  standard  procedure  for  these  analyses. 
In  some  cases  it  is  difficult  to  obtain  the  data  required  to 
perform  a  cost-benefit  analysis  and  to  evaluate  a  warranty. 
An  example  of  such  a  case  was  illustrated  in  this  research 


effort.  Unless  the  data  necessary  is  available  and  utilized 
to  perform  a  cost-benefit  analysis,  there  is  no  assurance 
the  Air  Force  is  purchasing  cost  effective  warranties. 

This  thesis  effort  is  useful  in  that  a  structure  for 
warranty  cost-benefit  analysis  is  provided  and  several 
problem  areas  are  identified.  Future  improvements  will 
require  data  collected  and  formatted  in  a  way  that  can  be 
used  to  perform  a  cost-benefit  analysis  prior  to  applying 
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